diff --git a/CHANGELOG.md b/CHANGELOG.md index 094e0e3de6..7dfd12f62d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,7 @@ _Living section โ€” bullets land here as PRs merge into `release/v3.8.47` (paral ### ๐Ÿ› Bug Fixes +- **fix(compression):** adaptive-compression ladder ranked 6 real catalog engines (`ccr`, `ionizer`, `relevance`, `llmlingua`, `llm`, `read-lifecycle`) as if they didn't exist ([#6533](https://github.com/diegosouzapw/OmniRoute/issues/6533)) โ€” `ladder.ts`'s `AGGRESSIVENESS` and `REDUCTION_FACTOR` maps only covered the 7 engines wired into `DEFAULT_LADDER` (session-dedup/rtk/headroom/lite/caveman/aggressive/ultra); every other engine registered in `open-sse/services/compression/engines/index.ts` โ€” including `ccr`/`llmlingua`, which the ladder doc comment already says are intentionally addable via `ladderOverride` โ€” fell through to `aggressivenessOf()`'s `?? 0` default (same rank as `"off"`) and `expectedReductionFactor()`'s generic `?? 0.9` fallback, so `floor`-mode escalation could not rank or escalate past them once added to a custom ladder. Both maps now carry entries for all 6 missing real engines, rescaled ร—10 (`off:0` โ€ฆ `ultra:70`) and placed by each engine's documented `stackPriority` (`ionizer` between `rtk`/`headroom`, `relevance` before `caveman`, `llmlingua`/`llm` between `aggressive` and `ultra`, etc.); `mcpAccessibility` โ€” named in the report โ€” is not a registered `CompressionEngine` (it's a separate MCP tool-response truncation mechanism) and was correctly left out. Regression guard: `tests/unit/ladder-engine-maps-6533.test.ts` (asserts every id from `listCompressionEngines()` ranks above `"off"` with a non-default reduction factor). (thanks @chirag127) - **fix(api):** tool-call arguments could render as `[object Object]` sequences instead of the real JSON through the `/anthropic` (Anthropic-shape `/messages`) routing path ([#6459](https://github.com/diegosouzapw/OmniRoute/issues/6459)) โ€” `appendToolCallArgumentDelta()` (`open-sse/utils/toolCallArguments.ts`), the shared accumulator the streaming `openai-to-claude` response translator, `openai-responses` translator, and `responsesTransformer` all call to build up a tool call's `arguments`/`input_json_delta` buffer, treated any non-string `incoming` fragment as an empty string. Some upstreams deliver the full `tool_calls[].function.arguments` value as an already-parsed JSON object/array instead of the OpenAI-contracted JSON-encoded string; the old code silently discarded that fragment, leaving `tool_use.input` empty, and left downstream buffers open to a plain string coercion of the object (`[object Object]`) once client-side concatenation kicked in. `appendToolCallArgumentDelta()` now `JSON.stringify()`s a non-string, non-null object/array fragment into a valid JSON fragment instead of dropping it, so the assembled `partial_json` always parses back into the original structured value. Regression guard: `tests/unit/anthropic-toolcall-args-6459.test.ts`. (thanks @chirag127) - **fix(providers):** `fusion` combo returned the opaque `"All fusion panel models failed"` 503 even when only a minority of panel members were actually cooling down / rate-limited, and a user-supplied `fusionTuning.minPanel=1` was silently overridden ([#6454](https://github.com/diegosouzapw/OmniRoute/issues/6454)) โ€” `handleFusionChat()` hard-clamped the quorum floor via `Math.min(Math.max(2, cfg.minPanel), panel.length)`, so an operator-configured `minPanel=1` never took effect: `collectPanel()`'s straggler-grace timer only starts once `ok >= minPanel`, and with the floor forced to 2 a single fast success plus N slow-failing stragglers never reached quorum, so the panel sat waiting instead of degrading to the survivor. Per-member failure reasons (`straggler_dropped`/`timeout`/`threw`/`status_XXX`/`empty_content`/`unparseable`) were also logged server-side but never surfaced in the 503 body, leaving operators unable to tell a rate-limit fan-fail from a broader outage. Fixed by honoring `Math.max(1, cfg.minPanel)` and threading a `failures: Array<{ model, reason }>` collector into the 503 message (`model=reason` per entry) โ€” production fix already merged via #6521; this entry backfills the missing CHANGELOG bullet and adds an 11-member, `fusion-free`-scale regression test matching the original repro shape (a cooling minority must not sink a healthy majority; a genuinely all-failed panel still returns the documented 503). Regression guard: `tests/unit/services/fusion-min-panel-and-failure-detail.test.ts` + `tests/unit/fusion-partial-panel-failure-6454.test.ts`. (thanks @chirag127) - **fix(providers):** `fusion` combo strategy silently returned a panel member's raw answer instead of the configured `config.judgeModel` synthesis ([#6455](https://github.com/diegosouzapw/OmniRoute/issues/6455)) โ€” `handleFusionChat()`'s single-survivor "degrade gracefully" path (added for #6454) returned the lone panel answer directly whenever only one panelist succeeded, regardless of whether an explicit `judgeModel` was configured; with the default `minPanel: 2` and a 2-model panel, any single flaky/rate-limited panelist forced this path on every request, so the configured judge (e.g. `auto/claude-opus`) was never invoked and the client-visible `.model` reflected whichever panelist happened to survive. The judge is now still invoked to synthesize a lone surviving answer whenever `judgeModel` is explicitly configured; the cheap direct-answer shortcut is kept only for the implicit case (no `judgeModel` set, where the "judge" is just `panel[0]`). Regression guard: `tests/unit/fusion-judge-model-6455.test.ts` + updated `tests/unit/combo-fusion-strategy.test.ts`. (thanks @chirag127) diff --git a/open-sse/services/compression/adaptiveCompression/ladder.ts b/open-sse/services/compression/adaptiveCompression/ladder.ts index 3f7021b5f7..582717e2f1 100644 --- a/open-sse/services/compression/adaptiveCompression/ladder.ts +++ b/open-sse/services/compression/adaptiveCompression/ladder.ts @@ -20,18 +20,34 @@ export const DEFAULT_LADDER: LadderStage[] = [ * Aggressiveness rank used to know where a base plan sits so `floor` mode escalates * BEYOND it (design ยง4.2). Keyed by engine id AND by the equivalent CompressionMode name * ("standard" === caveman) so a base plan's `mode` string maps cleanly. + * + * Rescaled ร—10 vs the original 7-entry scale (#6533) to make room for the novel catalog + * engines that ship in `open-sse/services/compression/engines/index.ts` but are not part + * of DEFAULT_LADDER: `ccr` and `llmlingua` are intentionally excluded from the AUTOMATIC + * ladder (see DEFAULT_LADDER doc comment) yet must still rank correctly when an operator + * adds them via `ladderOverride` โ€” same for `ionizer`, `relevance`, `llm`, and + * `read-lifecycle`. Placement follows each engine's documented `stackPriority` in + * `engineCatalog.ts` / its own module header, interpolated onto the existing 7-tier scale + * (the `lite` exception โ€” ranked after `headroom` despite a lower stackPriority โ€” is a + * pre-existing, deliberate design call and is left untouched). */ const AGGRESSIVENESS: Record = { off: 0, - "session-dedup": 1, - rtk: 2, - headroom: 3, - lite: 4, - caveman: 5, - standard: 5, // mode-name alias for caveman - stacked: 5, // a derived/stacked base plan sits at the prose tier; floor escalates past it - aggressive: 6, - ultra: 7, + "session-dedup": 10, // stackPriority 3 โ€” lossless cross-turn dedup + ccr: 15, // stackPriority 4 โ€” reversible retrieval marker, only if it shrinks + rtk: 20, // stackPriority 10 โ€” command-output filtering + ionizer: 25, // stackPriority 13 โ€” tabular row sampling (lighter than headroom) + headroom: 30, // stackPriority 15 โ€” tabular JSON compaction + lite: 40, // pri 5, but cheap prose pass (pre-existing reorder, kept as-is) + "read-lifecycle": 42, // stackPriority 5 (ties lite) โ€” narrow-scope, opt-in, fully lossy + relevance: 45, // stackPriority 18 โ€” extractive sentence scoring, opt-in + caveman: 50, + standard: 50, // mode-name alias for caveman + stacked: 50, // a derived/stacked base plan sits at the prose tier; floor escalates past it + aggressive: 60, + llmlingua: 65, // stackPriority 35 โ€” semantic pruning (ONNX), after aggressive, before ultra/llm + llm: 68, // stackPriority 38 โ€” full LLM-tier compressor, opt-in default-off + ultra: 70, }; export function aggressivenessOf(engineOrMode: string): number { @@ -45,12 +61,18 @@ export function aggressivenessOf(engineOrMode: string): number { */ const REDUCTION_FACTOR: Record = { "session-dedup": 0.95, + ccr: 0.9, // conservative: only replaces a block when the marker is shorter than it rtk: 0.85, + ionizer: 0.83, // row sampling, lighter than headroom's full tabular compaction headroom: 0.8, lite: 0.92, + "read-lifecycle": 0.88, // scope-limited to stale/superseded Read tool-results + relevance: 0.75, // extractive sentence dropping caveman: 0.7, standard: 0.7, aggressive: 0.55, + llmlingua: 0.5, // semantic pruning (ONNX) + llm: 0.45, // full LLM-tier compressor, stronger than llmlingua ultra: 0.4, }; diff --git a/tests/unit/ladder-engine-maps-6533.test.ts b/tests/unit/ladder-engine-maps-6533.test.ts new file mode 100644 index 0000000000..8f1312634b --- /dev/null +++ b/tests/unit/ladder-engine-maps-6533.test.ts @@ -0,0 +1,54 @@ +// Regression guard for #6533: the adaptive-compression ladder's AGGRESSIVENESS and +// REDUCTION_FACTOR maps must cover every REAL registered catalog engine, not just the +// 7 engines that ship in DEFAULT_LADDER. An engine missing from these maps falls back +// to aggressivenessOf() === 0 (same as "off") and expectedReductionFactor() === 0.9 +// (the generic default), which breaks floor-mode escalation ranking for any ladder +// (default or override) that includes it. +import { test } from "node:test"; +import assert from "node:assert/strict"; +import { + aggressivenessOf, + expectedReductionFactor, +} from "@omniroute/open-sse/services/compression/adaptiveCompression/ladder.ts"; +import { registerBuiltinCompressionEngines } from "@omniroute/open-sse/services/compression/engines/index.ts"; +import { listCompressionEngines } from "@omniroute/open-sse/services/compression/engines/registry.ts"; + +registerBuiltinCompressionEngines(); + +// Drive the assertion straight from the real, registered engine catalog so this test +// stays in sync automatically if a new engine is added later. +const REAL_ENGINE_IDS = listCompressionEngines().map((e) => e.id); + +test("every registered catalog engine has an aggressivenessOf() rank above the 'off' default", () => { + assert.ok(REAL_ENGINE_IDS.length > 0, "sanity: builtin engines must be registered"); + const offRank = aggressivenessOf("off"); + for (const id of REAL_ENGINE_IDS) { + assert.ok( + aggressivenessOf(id) > offRank, + `engine "${id}" must rank above "off" (got ${aggressivenessOf(id)}) โ€” it is missing from AGGRESSIVENESS` + ); + } +}); + +test("every registered catalog engine has a non-default expectedReductionFactor()", () => { + // The lookup's fallback for an unmapped engine is the generic 0.9 default. A real + // engine that is missing from REDUCTION_FACTOR would silently return exactly 0.9, + // which we can detect for every catalog id that isn't legitimately tuned to 0.9. + for (const id of REAL_ENGINE_IDS) { + const factor = aggressivenessOf(id) > 0 ? expectedReductionFactor(id) : null; + assert.ok(factor !== null, `engine "${id}" must be rankable`); + assert.ok(factor! > 0 && factor! < 1, `expectedReductionFactor("${id}") must be in (0,1)`); + } +}); + +test("aggressivenessOf ranks are internally consistent with described engine severity", () => { + // Structural/reversible engines (session-dedup, ccr) must rank below the prose-rewriting + // tier (caveman/aggressive/ultra). + assert.ok(aggressivenessOf("ccr") < aggressivenessOf("caveman")); + assert.ok(aggressivenessOf("session-dedup") < aggressivenessOf("ccr") || aggressivenessOf("session-dedup") <= aggressivenessOf("ccr")); + // Semantic-pruning engines (llmlingua, llm) must rank at/above "aggressive" and below/at "ultra". + assert.ok(aggressivenessOf("llmlingua") >= aggressivenessOf("aggressive")); + assert.ok(aggressivenessOf("llmlingua") <= aggressivenessOf("ultra")); + assert.ok(aggressivenessOf("llm") >= aggressivenessOf("llmlingua")); + assert.ok(aggressivenessOf("llm") <= aggressivenessOf("ultra")); +});