mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-03 13:52:09 +03:00
The fusion quorum-clamp/failure-detail root cause reported in #6454 was already fixed and merged via #6521 (open-sse/services/fusion.ts already carries Math.max(1, cfg.minPanel) + per-member failure reasons on this branch). That merge never landed a CHANGELOG bullet for #6454 itself. Backfills the missing bullet and adds a regression test at the exact repro scale (11-member fusion-free-style panel, 2 cooling / 9 healthy) to lock in that a cooling minority no longer sinks a healthy majority, while a genuinely all-failed panel still returns the documented 503.
This commit is contained in:
committed by
GitHub
parent
f4cd3e8c80
commit
533016af36
@@ -11,6 +11,7 @@ _Living section — bullets land here as PRs merge into `release/v3.8.47` (paral
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
- **fix(api):** tool-call arguments could render as `[object Object]` sequences instead of the real JSON through the `/anthropic` (Anthropic-shape `/messages`) routing path ([#6459](https://github.com/diegosouzapw/OmniRoute/issues/6459)) — `appendToolCallArgumentDelta()` (`open-sse/utils/toolCallArguments.ts`), the shared accumulator the streaming `openai-to-claude` response translator, `openai-responses` translator, and `responsesTransformer` all call to build up a tool call's `arguments`/`input_json_delta` buffer, treated any non-string `incoming` fragment as an empty string. Some upstreams deliver the full `tool_calls[].function.arguments` value as an already-parsed JSON object/array instead of the OpenAI-contracted JSON-encoded string; the old code silently discarded that fragment, leaving `tool_use.input` empty, and left downstream buffers open to a plain string coercion of the object (`[object Object]`) once client-side concatenation kicked in. `appendToolCallArgumentDelta()` now `JSON.stringify()`s a non-string, non-null object/array fragment into a valid JSON fragment instead of dropping it, so the assembled `partial_json` always parses back into the original structured value. Regression guard: `tests/unit/anthropic-toolcall-args-6459.test.ts`. (thanks @chirag127)
|
||||
- **fix(providers):** `fusion` combo returned the opaque `"All fusion panel models failed"` 503 even when only a minority of panel members were actually cooling down / rate-limited, and a user-supplied `fusionTuning.minPanel=1` was silently overridden ([#6454](https://github.com/diegosouzapw/OmniRoute/issues/6454)) — `handleFusionChat()` hard-clamped the quorum floor via `Math.min(Math.max(2, cfg.minPanel), panel.length)`, so an operator-configured `minPanel=1` never took effect: `collectPanel()`'s straggler-grace timer only starts once `ok >= minPanel`, and with the floor forced to 2 a single fast success plus N slow-failing stragglers never reached quorum, so the panel sat waiting instead of degrading to the survivor. Per-member failure reasons (`straggler_dropped`/`timeout`/`threw`/`status_XXX`/`empty_content`/`unparseable`) were also logged server-side but never surfaced in the 503 body, leaving operators unable to tell a rate-limit fan-fail from a broader outage. Fixed by honoring `Math.max(1, cfg.minPanel)` and threading a `failures: Array<{ model, reason }>` collector into the 503 message (`model=reason` per entry) — production fix already merged via #6521; this entry backfills the missing CHANGELOG bullet and adds an 11-member, `fusion-free`-scale regression test matching the original repro shape (a cooling minority must not sink a healthy majority; a genuinely all-failed panel still returns the documented 503). Regression guard: `tests/unit/services/fusion-min-panel-and-failure-detail.test.ts` + `tests/unit/fusion-partial-panel-failure-6454.test.ts`. (thanks @chirag127)
|
||||
- **fix(providers):** `fusion` combo strategy silently returned a panel member's raw answer instead of the configured `config.judgeModel` synthesis ([#6455](https://github.com/diegosouzapw/OmniRoute/issues/6455)) — `handleFusionChat()`'s single-survivor "degrade gracefully" path (added for #6454) returned the lone panel answer directly whenever only one panelist succeeded, regardless of whether an explicit `judgeModel` was configured; with the default `minPanel: 2` and a 2-model panel, any single flaky/rate-limited panelist forced this path on every request, so the configured judge (e.g. `auto/claude-opus`) was never invoked and the client-visible `.model` reflected whichever panelist happened to survive. The judge is now still invoked to synthesize a lone surviving answer whenever `judgeModel` is explicitly configured; the cheap direct-answer shortcut is kept only for the implicit case (no `judgeModel` set, where the "judge" is just `panel[0]`). Regression guard: `tests/unit/fusion-judge-model-6455.test.ts` + updated `tests/unit/combo-fusion-strategy.test.ts`. (thanks @chirag127)
|
||||
- **fix(providers):** image/diffusion models discovered from an upstream catalog (e.g. HuggingFace's live `/v1/models`) are no longer advertised as chat models ([#6457](https://github.com/diegosouzapw/OmniRoute/issues/6457)) — the chat catalog builder defaulted synced models with no modality info to `endpoints: ["chat"]`, so `huggingface/stabilityai/stable-diffusion-xl-base-1.0` showed up in the chat `/v1/models` listing and returned `400 "not a chat model"` when called. `catalog.ts` now skips any synced model already registered as an image model for that provider (via the new `isRegisteredImageModel()`), leaving `getAllImageModels()` to list it with the correct `type: "image"`. Regression guard: `tests/unit/image-model-not-in-chat-catalog-6457.test.ts`.
|
||||
- **fix(test):** replace the bare `expect(true).toBe(true)` tautology in `playground-api-tab.test.tsx`'s SSE test and close the `check:test-masking` gap that let it slip through for a full cycle ([#6404](https://github.com/diegosouzapw/OmniRoute/issues/6404)) — a prior pass (#6548) had already swapped the literal to `expect(sendBtn).toBeDefined()`, but that stayed just as vacuous: the test's fetch mock returned an empty `/v1/models` list, so `ApiTab`'s Send button is always `disabled` (`!selectedModel`) and the SSE branch never runs — the "SSE infra is verified" comment was never true. The test now mocks a real model, drives the model `<select>` to enable Send, asserts `sendBtn.disabled === false` before clicking, and asserts the streamed SSE delta (`"Hello!"`) actually reached the response editor. Root cause on the detector side: `check-test-masking.mjs`'s tautology subcheck only compares base-vs-HEAD counts within a PR's own diff (`headExtTaut > baseExtTaut`) and no-ops locally when `GITHUB_BASE_SHA`/`GITHUB_BASE_REF` are unset ("sem base ref — pulando") — so a tautology merged once, or checked with a bare local run, was invisible forever after. Added a new always-on, PR-independent absolute-floor scan (`scanBareTautologies` + `countBareTautologies`) over every git-tracked test file for the bare `expect(true).toBe(true)` / `assert.equal(1,1)` / `assert.strictEqual(1,1)` patterns specifically (deliberately excluding `assert.ok(true)`, which has ~15 pre-existing verified-legitimate try/catch-fallback uses repo-wide and stays governed by the lenient diff-only subcheck) — verified zero pre-existing hits repo-wide once this file was fixed, so the new floor is safe to enforce unconditionally. Regression guard: `tests/unit/check-test-masking.test.ts` (new `scanBareTautologies`/`countBareTautologies` cases) + `tests/unit/ui/playground-api-tab.test.tsx`. (thanks @chirag127)
|
||||
|
||||
100
tests/unit/fusion-partial-panel-failure-6454.test.ts
Normal file
100
tests/unit/fusion-partial-panel-failure-6454.test.ts
Normal file
@@ -0,0 +1,100 @@
|
||||
/**
|
||||
* Regression test for issue #6454 at the exact panel scale from the report
|
||||
* (11 panel members, `fusionTuning.minPanel=1`).
|
||||
*
|
||||
* #6454 reported that a fusion panel returned the opaque "All fusion panel
|
||||
* models failed" error even though only a minority of members were actually
|
||||
* cooling down / rate-limited — the majority would have answered given the
|
||||
* chance. Root cause (fixed by #6521, merged into this branch already):
|
||||
* `open-sse/services/fusion.ts` used to hard-clamp the quorum floor to
|
||||
* `Math.max(2, cfg.minPanel)`, silently overriding a user-supplied
|
||||
* `minPanel=1` and per-member failure reasons were never surfaced in the
|
||||
* 503 body.
|
||||
*
|
||||
* This file exercises the scenario at the reported scale (11 members) to
|
||||
* lock in the fix as a permanent regression guard: a minority cooling down
|
||||
* must not sink an otherwise-healthy majority, and a genuinely all-failed
|
||||
* panel must still return the documented 503.
|
||||
*/
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import { handleFusionChat } from "../../open-sse/services/fusion.ts";
|
||||
|
||||
const noop = () => {};
|
||||
const log = { info: noop, warn: noop, debug: noop, error: noop };
|
||||
|
||||
type Body = Record<string, unknown>;
|
||||
|
||||
function okResponse(content: string): Promise<Response> {
|
||||
const body = JSON.stringify({ choices: [{ message: { role: "assistant", content } }] });
|
||||
return Promise.resolve(
|
||||
new Response(body, { status: 200, headers: { "Content-Type": "application/json" } })
|
||||
);
|
||||
}
|
||||
|
||||
function errResponse(status: number): Promise<Response> {
|
||||
const body = JSON.stringify({ error: { message: "boom" } });
|
||||
return Promise.resolve(new Response(body, { status, headers: { "Content-Type": "application/json" } }));
|
||||
}
|
||||
|
||||
// Mirrors the #6454 repro: an 11-member "fusion-free" style panel where only
|
||||
// 2 members are actually cooling/rate-limited and 9 are healthy.
|
||||
const PANEL_11 = [
|
||||
"auto/claude-opus",
|
||||
"auto/gpt-5.5",
|
||||
"auto/sonar-pro",
|
||||
"auto/deepseek-v4",
|
||||
"auto/minimax-m3",
|
||||
"auto/glm-5.2",
|
||||
"auto/zai-glm-4.7",
|
||||
"auto/mimo-v2.5",
|
||||
"auto/gemma-4-31b",
|
||||
"auto/llama-3.3-70b",
|
||||
"auto/llama-3.1-8b",
|
||||
];
|
||||
const COOLING = new Set(["auto/glm-5.2", "auto/zai-glm-4.7"]);
|
||||
|
||||
test("fusion #6454: a cooling minority (2/11) does not sink a healthy majority — panel proceeds, not 'all failed'", async () => {
|
||||
const seen: string[] = [];
|
||||
const handleSingleModel = (_b: Body, m: string) => {
|
||||
seen.push(m);
|
||||
if (COOLING.has(m)) return errResponse(429);
|
||||
return okResponse(`ans-${m}`);
|
||||
};
|
||||
|
||||
const res = await handleFusionChat({
|
||||
body: { messages: [{ role: "user", content: "List 3 file operations" }] },
|
||||
models: PANEL_11,
|
||||
handleSingleModel,
|
||||
log,
|
||||
judgeModel: "auto/claude-opus",
|
||||
tuning: { minPanel: 1, stragglerGraceMs: 4000, panelHardTimeoutMs: 60000 },
|
||||
});
|
||||
|
||||
assert.notEqual(res.status, 503, "9/11 healthy members must not be reported as a total panel failure");
|
||||
const body = (await res.clone().json()) as { choices?: Array<{ message?: { content?: string } }> };
|
||||
const text = body.choices?.[0]?.message?.content ?? "";
|
||||
assert.ok(text.length > 0, "should carry a real synthesized/answer body, not an error");
|
||||
// The judge call is the final dispatch, invoked with every healthy answer available to it.
|
||||
assert.equal(seen[seen.length - 1], "auto/claude-opus");
|
||||
});
|
||||
|
||||
test("fusion #6454: a genuinely all-failed 11-member panel still returns the documented 503 (no over-correction)", async () => {
|
||||
const handleSingleModel = (_b: Body, m: string) => {
|
||||
return COOLING.has(m) ? errResponse(429) : errResponse(500);
|
||||
};
|
||||
|
||||
const res = await handleFusionChat({
|
||||
body: { messages: [{ role: "user", content: "List 3 file operations" }] },
|
||||
models: PANEL_11,
|
||||
handleSingleModel,
|
||||
log,
|
||||
judgeModel: "auto/claude-opus",
|
||||
tuning: { minPanel: 1, stragglerGraceMs: 4000, panelHardTimeoutMs: 60000 },
|
||||
});
|
||||
|
||||
assert.equal(res.status, 503, "a genuinely all-failed panel must still surface the fusion failure error");
|
||||
const body = (await res.clone().json()) as { error: { message: string } };
|
||||
assert.match(body.error.message, /All fusion panel models failed/);
|
||||
});
|
||||
Reference in New Issue
Block a user