fix(providers): backfill #6454 CHANGELOG bullet + 11-member fusion regression guard (#6614)

The fusion quorum-clamp/failure-detail root cause reported in #6454 was
already fixed and merged via #6521 (open-sse/services/fusion.ts already
carries Math.max(1, cfg.minPanel) + per-member failure reasons on this
branch). That merge never landed a CHANGELOG bullet for #6454 itself.

Backfills the missing bullet and adds a regression test at the exact
repro scale (11-member fusion-free-style panel, 2 cooling / 9 healthy)
to lock in that a cooling minority no longer sinks a healthy majority,
while a genuinely all-failed panel still returns the documented 503.
This commit is contained in:
Diego Rodrigues de Sa e Souza
2026-07-07 22:38:26 -03:00
committed by GitHub
parent f4cd3e8c80
commit 533016af36
2 changed files with 101 additions and 0 deletions

View File

@@ -11,6 +11,7 @@ _Living section — bullets land here as PRs merge into `release/v3.8.47` (paral
### 🐛 Bug Fixes
- **fix(api):** tool-call arguments could render as `[object Object]` sequences instead of the real JSON through the `/anthropic` (Anthropic-shape `/messages`) routing path ([#6459](https://github.com/diegosouzapw/OmniRoute/issues/6459)) — `appendToolCallArgumentDelta()` (`open-sse/utils/toolCallArguments.ts`), the shared accumulator the streaming `openai-to-claude` response translator, `openai-responses` translator, and `responsesTransformer` all call to build up a tool call's `arguments`/`input_json_delta` buffer, treated any non-string `incoming` fragment as an empty string. Some upstreams deliver the full `tool_calls[].function.arguments` value as an already-parsed JSON object/array instead of the OpenAI-contracted JSON-encoded string; the old code silently discarded that fragment, leaving `tool_use.input` empty, and left downstream buffers open to a plain string coercion of the object (`[object Object]`) once client-side concatenation kicked in. `appendToolCallArgumentDelta()` now `JSON.stringify()`s a non-string, non-null object/array fragment into a valid JSON fragment instead of dropping it, so the assembled `partial_json` always parses back into the original structured value. Regression guard: `tests/unit/anthropic-toolcall-args-6459.test.ts`. (thanks @chirag127)
- **fix(providers):** `fusion` combo returned the opaque `"All fusion panel models failed"` 503 even when only a minority of panel members were actually cooling down / rate-limited, and a user-supplied `fusionTuning.minPanel=1` was silently overridden ([#6454](https://github.com/diegosouzapw/OmniRoute/issues/6454)) — `handleFusionChat()` hard-clamped the quorum floor via `Math.min(Math.max(2, cfg.minPanel), panel.length)`, so an operator-configured `minPanel=1` never took effect: `collectPanel()`'s straggler-grace timer only starts once `ok >= minPanel`, and with the floor forced to 2 a single fast success plus N slow-failing stragglers never reached quorum, so the panel sat waiting instead of degrading to the survivor. Per-member failure reasons (`straggler_dropped`/`timeout`/`threw`/`status_XXX`/`empty_content`/`unparseable`) were also logged server-side but never surfaced in the 503 body, leaving operators unable to tell a rate-limit fan-fail from a broader outage. Fixed by honoring `Math.max(1, cfg.minPanel)` and threading a `failures: Array<{ model, reason }>` collector into the 503 message (`model=reason` per entry) — production fix already merged via #6521; this entry backfills the missing CHANGELOG bullet and adds an 11-member, `fusion-free`-scale regression test matching the original repro shape (a cooling minority must not sink a healthy majority; a genuinely all-failed panel still returns the documented 503). Regression guard: `tests/unit/services/fusion-min-panel-and-failure-detail.test.ts` + `tests/unit/fusion-partial-panel-failure-6454.test.ts`. (thanks @chirag127)
- **fix(providers):** `fusion` combo strategy silently returned a panel member's raw answer instead of the configured `config.judgeModel` synthesis ([#6455](https://github.com/diegosouzapw/OmniRoute/issues/6455)) — `handleFusionChat()`'s single-survivor "degrade gracefully" path (added for #6454) returned the lone panel answer directly whenever only one panelist succeeded, regardless of whether an explicit `judgeModel` was configured; with the default `minPanel: 2` and a 2-model panel, any single flaky/rate-limited panelist forced this path on every request, so the configured judge (e.g. `auto/claude-opus`) was never invoked and the client-visible `.model` reflected whichever panelist happened to survive. The judge is now still invoked to synthesize a lone surviving answer whenever `judgeModel` is explicitly configured; the cheap direct-answer shortcut is kept only for the implicit case (no `judgeModel` set, where the "judge" is just `panel[0]`). Regression guard: `tests/unit/fusion-judge-model-6455.test.ts` + updated `tests/unit/combo-fusion-strategy.test.ts`. (thanks @chirag127)
- **fix(providers):** image/diffusion models discovered from an upstream catalog (e.g. HuggingFace's live `/v1/models`) are no longer advertised as chat models ([#6457](https://github.com/diegosouzapw/OmniRoute/issues/6457)) — the chat catalog builder defaulted synced models with no modality info to `endpoints: ["chat"]`, so `huggingface/stabilityai/stable-diffusion-xl-base-1.0` showed up in the chat `/v1/models` listing and returned `400 "not a chat model"` when called. `catalog.ts` now skips any synced model already registered as an image model for that provider (via the new `isRegisteredImageModel()`), leaving `getAllImageModels()` to list it with the correct `type: "image"`. Regression guard: `tests/unit/image-model-not-in-chat-catalog-6457.test.ts`.
- **fix(test):** replace the bare `expect(true).toBe(true)` tautology in `playground-api-tab.test.tsx`'s SSE test and close the `check:test-masking` gap that let it slip through for a full cycle ([#6404](https://github.com/diegosouzapw/OmniRoute/issues/6404)) — a prior pass (#6548) had already swapped the literal to `expect(sendBtn).toBeDefined()`, but that stayed just as vacuous: the test's fetch mock returned an empty `/v1/models` list, so `ApiTab`'s Send button is always `disabled` (`!selectedModel`) and the SSE branch never runs — the "SSE infra is verified" comment was never true. The test now mocks a real model, drives the model `<select>` to enable Send, asserts `sendBtn.disabled === false` before clicking, and asserts the streamed SSE delta (`"Hello!"`) actually reached the response editor. Root cause on the detector side: `check-test-masking.mjs`'s tautology subcheck only compares base-vs-HEAD counts within a PR's own diff (`headExtTaut > baseExtTaut`) and no-ops locally when `GITHUB_BASE_SHA`/`GITHUB_BASE_REF` are unset ("sem base ref — pulando") — so a tautology merged once, or checked with a bare local run, was invisible forever after. Added a new always-on, PR-independent absolute-floor scan (`scanBareTautologies` + `countBareTautologies`) over every git-tracked test file for the bare `expect(true).toBe(true)` / `assert.equal(1,1)` / `assert.strictEqual(1,1)` patterns specifically (deliberately excluding `assert.ok(true)`, which has ~15 pre-existing verified-legitimate try/catch-fallback uses repo-wide and stays governed by the lenient diff-only subcheck) — verified zero pre-existing hits repo-wide once this file was fixed, so the new floor is safe to enforce unconditionally. Regression guard: `tests/unit/check-test-masking.test.ts` (new `scanBareTautologies`/`countBareTautologies` cases) + `tests/unit/ui/playground-api-tab.test.tsx`. (thanks @chirag127)

View File

@@ -0,0 +1,100 @@
/**
* Regression test for issue #6454 at the exact panel scale from the report
* (11 panel members, `fusionTuning.minPanel=1`).
*
* #6454 reported that a fusion panel returned the opaque "All fusion panel
* models failed" error even though only a minority of members were actually
* cooling down / rate-limited — the majority would have answered given the
* chance. Root cause (fixed by #6521, merged into this branch already):
* `open-sse/services/fusion.ts` used to hard-clamp the quorum floor to
* `Math.max(2, cfg.minPanel)`, silently overriding a user-supplied
* `minPanel=1` and per-member failure reasons were never surfaced in the
* 503 body.
*
* This file exercises the scenario at the reported scale (11 members) to
* lock in the fix as a permanent regression guard: a minority cooling down
* must not sink an otherwise-healthy majority, and a genuinely all-failed
* panel must still return the documented 503.
*/
import test from "node:test";
import assert from "node:assert/strict";
import { handleFusionChat } from "../../open-sse/services/fusion.ts";
const noop = () => {};
const log = { info: noop, warn: noop, debug: noop, error: noop };
type Body = Record<string, unknown>;
function okResponse(content: string): Promise<Response> {
const body = JSON.stringify({ choices: [{ message: { role: "assistant", content } }] });
return Promise.resolve(
new Response(body, { status: 200, headers: { "Content-Type": "application/json" } })
);
}
function errResponse(status: number): Promise<Response> {
const body = JSON.stringify({ error: { message: "boom" } });
return Promise.resolve(new Response(body, { status, headers: { "Content-Type": "application/json" } }));
}
// Mirrors the #6454 repro: an 11-member "fusion-free" style panel where only
// 2 members are actually cooling/rate-limited and 9 are healthy.
const PANEL_11 = [
"auto/claude-opus",
"auto/gpt-5.5",
"auto/sonar-pro",
"auto/deepseek-v4",
"auto/minimax-m3",
"auto/glm-5.2",
"auto/zai-glm-4.7",
"auto/mimo-v2.5",
"auto/gemma-4-31b",
"auto/llama-3.3-70b",
"auto/llama-3.1-8b",
];
const COOLING = new Set(["auto/glm-5.2", "auto/zai-glm-4.7"]);
test("fusion #6454: a cooling minority (2/11) does not sink a healthy majority — panel proceeds, not 'all failed'", async () => {
const seen: string[] = [];
const handleSingleModel = (_b: Body, m: string) => {
seen.push(m);
if (COOLING.has(m)) return errResponse(429);
return okResponse(`ans-${m}`);
};
const res = await handleFusionChat({
body: { messages: [{ role: "user", content: "List 3 file operations" }] },
models: PANEL_11,
handleSingleModel,
log,
judgeModel: "auto/claude-opus",
tuning: { minPanel: 1, stragglerGraceMs: 4000, panelHardTimeoutMs: 60000 },
});
assert.notEqual(res.status, 503, "9/11 healthy members must not be reported as a total panel failure");
const body = (await res.clone().json()) as { choices?: Array<{ message?: { content?: string } }> };
const text = body.choices?.[0]?.message?.content ?? "";
assert.ok(text.length > 0, "should carry a real synthesized/answer body, not an error");
// The judge call is the final dispatch, invoked with every healthy answer available to it.
assert.equal(seen[seen.length - 1], "auto/claude-opus");
});
test("fusion #6454: a genuinely all-failed 11-member panel still returns the documented 503 (no over-correction)", async () => {
const handleSingleModel = (_b: Body, m: string) => {
return COOLING.has(m) ? errResponse(429) : errResponse(500);
};
const res = await handleFusionChat({
body: { messages: [{ role: "user", content: "List 3 file operations" }] },
models: PANEL_11,
handleSingleModel,
log,
judgeModel: "auto/claude-opus",
tuning: { minPanel: 1, stragglerGraceMs: 4000, panelHardTimeoutMs: 60000 },
});
assert.equal(res.status, 503, "a genuinely all-failed panel must still surface the fusion failure error");
const body = (await res.clone().json()) as { error: { message: string } };
assert.match(body.error.message, /All fusion panel models failed/);
});