Merged into release/v3.8.51. Batch review caught that the slot-release replaced the #5923 recordComboFailure call on the no-executable-targets path (pin auto-clear would freeze); restored both effects side by side + regression guard in combo-routing-engine.test.ts (proven RED without the fix, GREEN after — 211/211 across the focused combo/quota/opencode battery, quota-share-strategy 31/31 on the final branch head). Thanks @oyi77!
Retargetado para release/v3.8.51 (release/v3.8.50 está congelada — freeze issue #11439). Validado em lote combinado (batch-0824h2, junto de #11435/#11436/#11437) contra o tip de release/v3.8.51: typecheck:core limpo, gates estáticos OK, 127/127 testes focados passando.
Investigação sólida com repro real via container isolado, três causas independentes identificadas e corrigidas com testes de regressão dedicados para cada uma. Obrigado pela contribuição!
Validated on the combined 12-PR batch board: combo-max-global-attempts-config 4/4 plus the neighboring combo suites green, typecheck:core clean, gates within baseline. clampGlobalAttempts mirrors clampComboDepth with a 200 hard cap, so a bad config can never disable the budget. Closes the configurability item from #11134. Thank you @pacocartones!
⭐5 — Fix(#10967,#10966): combo diag exhausted_connection truncava o UUID por hardcode de provider="unknown"; recovery hint de quota caía em "retry" genérico. TDD RED→GREEN, 149 testes-irmãos verdes. UNSTABLE é o base-red inherited #9985.
Validado + reconciliado: 86/86 testes focados (combo-disable-session-stickiness, base-executor-sanitize-effort, command-code-executor) passando. Incluí o rebaseline do file-size (commandCode.ts 1023→1038, crescimento legítimo deste PR) diretamente no branch — evitando o erro que cometi antes (rebaseline só na worktree local, nunca chegando ao branch real). Correção real de bug com repro ao vivo documentada. CI vermelho é o base-red já rastreado em #9985. Obrigado!
Reconciliado com a release (conflito aditivo em targetTimeoutRunner.ts — combina o warning G3 já mergeado com a resolução de effectiveTimeoutMs deste PR) e revalidado: 41/41 testes focados passando (upstream-timeout-connection-tier, combo-target-timeout-runner, provider-specific-data-schema). CI vermelho é o base-red já rastreado em #9985. Obrigado!
Obrigado — feature substancial e bem estruturada: separa qualidade operacional (comportamento de wire: 4xx/5xx, 429, respostas malformadas, stream interrompido) de qualidade semântica (só setada por avaliadores externos, nunca inferida do sucesso HTTP), com confidence/sample-awareness para não deixar poucos sucessos de sorte dominarem o ranking. Instrumentação de streaming (TTFT/ITL) threaded até RoutingEvent, endpoint de explicabilidade, e teste E2E determinístico cobrindo degradação→recuperação→blip.
Validação (worktree própria a partir de origin/release/v3.8.50, merge limpo, 0 conflitos):
- typecheck:core limpo, complexity/cognitive-complexity dentro do baseline
- 59/59 testes passando (mlx-provider, routing-adaptive-e2e, routing-events(-concurrency), routing-otel, routing-quality, routing-scoring-quality, stream-timing, auto-combo-scoring-clamp)
Merged — locally validated together with related cryptiklemur PRs (typecheck:core clean, complexity/cognitive/file-size/changelog gates green, focused tests passing). Real bug, clean fix, great regression test. Thanks!
Merged — per owner decision on RFC #10141: gateway token estimates (chars/4) become advisory-only for combo routing/context checks, never a hard pre-dispatch 400. Locally validated (31/31 focused tests across capability filtering, target resolution, and the #8841 repro), file-size/changelog gates clean, merges conflict-free against the current release tip. Thanks for the well-scoped fix and for flagging this as an RFC first!
Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
`open-sse/services/combo.ts` imported "../../src/lib/localDb.js" — a .js suffix
on a module that only exists as .ts. Turbopack resolved it by accident until the
dependency-tree change in #10647; after that the instrumentation hook died at boot
with MODULE_NOT_FOUND, breaking `npm run dev` and the production build (60
consecutive red `Build App` runs on release/v3.8.50).
Fixes the same latent pattern in src/lib/usage/usageLedger.ts, which survived only
because it is an `import type` and is erased before resolution.
Adds a guard rejecting relative .js specifiers across open-sse/ and src/. Package
specifiers are untouched: publishing ESM as .js is legitimate there (e.g.
@modelcontextprotocol/sdk), and only first-party relative imports are first-party
TypeScript.
Closes#10674
Co-authored-by: Xiangzhe <bakryun0718@proton.me>
* feat(admission): per-target lane-aware probes for combo/fusion fan-out (#9654 Wave 2)
Combo and fusion fan out N targets without ever consulting the adaptive-admission
layer: the parent request holds one lease, but each fan-out target is dispatched
unconditionally. With virtual lanes enabled (OMNIROUTE_CHAT_VIRTUAL_LANES=1), a
connection whose lane queue is full now SKIPS additional fan-out targets instead
of piling more queued work onto an already-congested session.
Adds PerTargetAdmissionHook (admission/types.ts) + createPerTargetAdmissionHook
factory (chatAdmission.ts): strictly non-blocking (maxWaitMs 0 - skip, never
queue), a no-op when virtual lanes are off, keyed to the parent tenantKey, and
release-on-admit so the probe is a capacity gate, not a hold.
Threaded through every parallel fan-out path:
- priority/weighted executeTarget + round-robin skip chains (combo.ts)
- fusion panel before fan-out (fusion.ts), judge fallback prefers survivors
- chaos parallel panel (autoCombo/chaosEngine.ts)
- tryFusionDispatch / tryRuntimeUnitDispatch / buildBaseOptions (dispatchPrelude.ts)
- chat.ts primary + safety-net redirect call sites
Snapshot exposes virtualLanes so the no-op gate is cheap and honest.
Tests: tests/unit/combo-lane-awareness-9654.test.ts (10 tests) - factory
semantics, priority/RR skip, fusion panel drop + all-skipped 503, no-hook
backward-compat baseline.
* feat(flags): activation UX - env-wins adaptive virtual-lanes flag + env docs (#9654 Wave 2)
U7: make adaptive virtual admission lanes discoverable + activatable.
- New OMNIROUTE_CHAT_VIRTUAL_LANES feature flag (boolean/runtime/requiresRestart) in featureFlagDefinitions + en.json i18n key.
- lib/admissionVirtualLanes.ts: env-wins resolver (env > DB > default) + boot warm folding a DB-sourced override into the process-global runtime env via reloadAdaptiveAdmissionRuntime(options.env) - no process.env mutation, no open-sse changes. Env still wins; DB toggle gates at next boot.
- GET /api/settings/feature-flags special-cases the flag to report the gate true source (ccDiscoveryAliases precedent); flagPayload helper dedupes the payload shape.
- Wire the warm into instrumentation-node registerNodejs (non-fatal, DB-ready).
- Document the master switch in .env.example + ENVIRONMENT.md with the system-1/system-2 distinction; zero new env-doc-sync drift.
- 11 new tests (resolver precedence + warm); 60/60 across feature-flag suites; typecheck core clean; ESLint + doc gates green.
* feat(mcp): surface adaptive admission lane data in omniroute_get_health (#9654 Wave 2)
U8: make adaptive virtual-lane admission visible to agents via the MCP health tool. handleGetHealth now surfaces a curated adaptiveAdmission block from the health payload (which already carried the runtime snapshot but was dropping it): virtualLanes/pressure/utilization/laneCount/laneQueuedCount/laneQueuedCost, laneTenants capped at top-10 by queued cost, admitted/rejected/wouldReject counts, shutdown. Block omitted entirely when the health endpoint reports none.
isLaneFlagOn mirrors the runtime 1|true convention so a string serialization can never invert a boolean lane report. getHealthOutput schema extended with the matching optional shape; tool description updated.
4 new dispatch tests (full block, top-10 cap/order, omission, defensive coercion of string flags + malformed lane entries) - 22/22 in essentialTools.test.ts. README: Adaptive Admission Lane Data table + Skills & Tool Navigability audit (29/43 schema entries covered, 14 undocumented, tool_search keyword runtime discovery, full catalog in docs/frameworks/MCP-SERVER.md).
No new lint errors (4 pre-existing in server.ts), typecheck core clean, doc counts + fabricated-docs gates green.
* docs: add changelog entry for #9654 Wave 2 (#10039)
* fix(codeql): suppress js/insufficient-password-hash false positive in lane-key fingerprinting (#10039)
resolveSessionId sha256-hashes bearer/x-api-key/x-goog-api-key to derive a deterministic, non-reversible per-key lane-bucket ID for virtual admission lanes (#9654). This is not password storage or verification, so the rule is a false positive; suppress it inline (same house style as src/lib/sync/tokens.ts) to clear the codeqlAlerts ratchet (2 > baseline 1) that blocks #10039 and every PR against release/v3.8.50.
* docs(mcp): complete MCP server README tool reference (#10039)
The MCP server README covered only 29 of the 43 schema entries, listing the
remaining tools solely as a gap note with omniroute_tool_search as the runtime
fallback. Add tool-reference tables for the agent-skills trio, oneproxy trio,
web_fetch/web_search, tool_search, create_combo, set_routing_strategy,
pick_fastest_model, sync_pricing, and db_health_check so the README covers the
full schemas catalog, and fold the coverage note into the tool_search discovery
paragraph.
* fix(chat): drop unused correlationId from safety-net combo redirect (#10039)
handleComboChat's HandleComboChatOptions has no correlationId member and
the combo pipeline never consumes it; the property was copied from the
handleSingleModelChat options shape by accident and introduced a new
TS2353 under the open-sse workspace typecheck gate.
* fix(i18n): translate featureFlagChatVirtualLanesEnabledDescription into 42 locales (#10039)
en.json gained the flag description in this PR but the locale catalogs
were never mirrored, failing the pt-BR key-parity (#6695) and vi
completeness gates. Adds a real translation to every locale, keeping the
zh-CN/zh-TW glossary canonical terms (提供者/儀表板) and no ICU drift.
* chore(quality): ratchet open-sse-typecheck baseline down (#10039)
The Wave 2 admission refactor removed 66 baselined open-sse type errors;
re-freeze the baseline so the gate pins the new, tighter state.
* docs: resync provider reference to 341 and CLI tools to 34
The release branch gained an 11th no-auth provider (freeaiapikey registry
resync, #10233) and a 26th CLI Code tool without regenerating the
auto-generated docs, leaving every PR against release/v3.8.50 failing the
Docs Gates strict validator (code 341 vs doc 340, CLI 34 vs "33 tools").
Regenerate docs/reference/PROVIDER_REFERENCE.md and sync the provider/tool
counts across README.md, AGENTS.md, llm.txt plus 42 i18n mirrors,
package.json description, and the four diagram SVGs.
* fix(tests): align count expectations with live catalogs (pre-existing release drift)
Release/v3.8.50 currently fails five gates on its own tree; this PR inherits
them. Fix the stale expectations to match live code:
- feature-flags-settings: 48 -> 49 flags (Wave 2 adds OMNIROUTE_CHAT_VIRTUAL_LANES)
- cli-tools-schema / cli-catalog-counts: 33 -> 34 tools (zcode added; 26 code = 21 visible + 5 none)
- optional-transformers-dependency: onnxruntime-node ~1.24.3 -> ~1.27.0 (bump #10382)
- stryker.conf.json: register chatcore-header-drop-warn-dedupe-10315 test
- check-public-creds: freeze zcodeProtocol clientId false positive (client identifier, not a credential)
* fix(tests): follow release's onnxruntime-node revert to ~1.24.3
release/v3.8.50's #10543 pinned onnxruntime-node back to ~1.24.3 after
#10403's ~1.27.0 bump caused npm to nest a second native copy under
@huggingface/transformers and broke the Docker SONAME contract. This
PR's own drift-alignment commit (57b9c033) predates that revert and
still expected ~1.27.0; the 3-way merge did not flag it as a textual
conflict since only one side touched this exact line, but the merged
tree became internally inconsistent (package.json ~1.24.3 vs test
expecting ~1.27.0). Align the test with the now-canonical release
value.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(quality): dedupe stryker.conf.json chatcore-header-drop-warn-dedupe entry
The 3-way merge applied both sides' insertion of the same test-file entry
at different positions, producing a duplicate with broken indentation.
Adopted release's clean version of the file.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: Brandon Bennett <brandonbennett@macbookair.myfiosgateway.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Co-authored-by: Brandon Bennett <branben@users.noreply.github.com>
* fix(combo): surface context-overflow before compression so oversized requests fail fast with a clear error (#10225)
* fix(combo): make context-overflow deferral target-aware for native Codex passthrough (#10225)
The deferral added by the prior commit checked only operator-named
compression exclusions when deciding whether at least one target "can
compress" — it never accounted for native Codex Responses passthrough
targets, which chatCore.ts unconditionally excludes from compression
(compressionExcluded = nativeCodexPassthrough || ...). Deferring on such
a target's account let an oversized request skip both the combo preflight
AND compression, reaching fetch() uncompressed.
Thread the same request-shape facts chatCore.ts uses
(shouldUseNativeCodexPassthrough: provider/sourceFormat/endpointPath/body/
headers) down into getKnownContextOverflow so the deferral decision can
never drift from chatCore's own — a native-codex-passthrough target now
never counts as "compressible", so a pool made only of such targets keeps
the fast local 400 instead of a wasted round trip.
Adds regression coverage: the pure getKnownContextOverflow target-aware
check, an end-to-end handleComboChat proof that a native-codex-only pool
fails fast with zero dispatches, and two real handleChatCore-path tests
proving compression actually reduces the dispatched body when eligible,
and that a still-too-large-after-compression request is rejected locally
without an upstream call.
---------
Co-authored-by: adevwithpurpose <adevwithpurpose@users.noreply.github.com>
* fix(resilience): keep combo quality and auth reasons separate and redact connection labels in terminal errors (#10314)
* fix(resilience): sanitize identifiers in error text, add explicit terminal-status policy, fix classifier ordering (#10314)
Four gaps in the prior combo-error-aggregation fix:
- formatComboOutcomes() only redacted connection identifiers in the model
label, never in the raw upstream error TEXT — a proxy echoing a
connection/account id back in its error body leaked it into the
client-facing terminal message. Redact both.
- The terminal HTTP status was still `lastStatus` — whichever target
happened to fail last, independent of the other targets' reasons. Add
resolveComboTerminalStatus(): preserve a 4xx only when every eligible
target's failure is genuinely "the request is invalid" (model-class);
a heterogeneous mix (e.g. a quality failure + a sibling's 401) now
normalizes to a 5xx-class status reflecting an infra/provider problem,
never a misleading client error borrowed from an unrelated target.
- classifyComboOutcome()'s ordering had `status === 408 || status >= 499`
checked before `status >= 500`, making the provider branch permanently
unreachable — every real 5xx (500/502/503/504) was silently mislabeled
as "timeout". Fixed to an exact match (408/499) and gave 429 its own
explicit `rate_limit` kind instead of falling into the generic "model"
(request-invalid) bucket by accident.
- Added an integration-level regression driving the real handleComboChat
wiring end-to-end (quality failure + sibling 401, and a success-after-
quality-failure case), not just the pure aggregation helpers.
Updated three pre-existing tests whose assertions encoded the OLD
last-writer-wins contract this fix intentionally supersedes (#8486 Part B
antigravity retryAfter tests, two combo-routing-engine status/message
tests) to the new, more precise contract; verified the underlying #8486
concern (wrong target's retryAfter header) is still honored under the new
status policy.
---------
Co-authored-by: adevwithpurpose <adevwithpurpose@users.noreply.github.com>
* feat(quota): Phase 2 adapters, reset timers, analytics, and dashboard API
* feat(routing): add quota-aware provider scheduling (opt-in)
* fix(db): rename migration to 148_provider_quota_state.sql
* fix(quota): harden quota state route, isolate phase2 tests, slim env diff
- route: requireManagementAuth + Zod body validation + buildErrorBody
sanitization (Hard Rule #12); fix clearProviderQuotaState -> clearProviderQuota
- .env.example/ENVIRONMENT.md: drop ~20 foreign vars, keep only
OMNIROUTE_QUOTA_AWARE_ROUTING (migration 148)
- tests/unit/quota-phase2.test.ts: DATA_DIR mkdtemp + resetDbInstance teardown
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* chore(ci): fix docs-sync + eslint-suppression drift for quota branch
CI gates flagged on PR #10126 head 43335f07:
- migration counts in README/AGENTS/llm.txt were stale (145 -> 146)
- regenerate docs/reference/PROVIDER_REFERENCE.md (gen-provider-reference)
- sync root llm.txt body into all 42 i18n mirrors (headers preserved)
- prune eslint suppressions that no longer occur
--no-verify: pre-commit docs-sync was failing on a pre-existing
release-base artifact (changelog 3.8.49 vs package 3.8.50) — fixed by
the changelog entry in the prior commit; re-verify in CI.
* chore(skills): regenerate agent skills (add omni-settings)
Merge-integrity CI gate flagged a missing generated skill. Regenerated
with check:agent-skills-sync --apply: +omni-settings, 45 unchanged.
* fix(ci): resolve Fast Quality Gates regressions on quota branch
- check-migration-numbering: migration 148 (provider_quota_state) landed
on this branch, so the KNOWN_GAPS allowlist entry is stale — remove it
(stale-enforcement 6A.3: 'REMOVA a entrada')
- open-sse/utils/stream.ts: duplicate sseCommentsEnabled import from a
bad merge (lines 31 + 77) — TS2300 duplicate identifier; drop the
duplicate so the open-sse typecheck gate is back within baseline
* docs: sync migration count to 149 after release merge
Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
* test(migrations): align 148 gap assertion after 148_provider_quota_state.sql landed
The phase-2 branch added 148_provider_quota_state.sql, and 148 was already
removed from KNOWN_GAPS in scripts/check/check-migration-numbering.mjs. The
frozen-allowlists assertion still expected 148 to be a gap, so it failed.
Flip the assertion to match the allowlist (same pattern as 143/147).
Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
---------
Co-authored-by: benzntech <benzntech@users.noreply.github.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Co-authored-by: adevwithpurpose <adevwithpurpose@users.noreply.github.com>
Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
* fix(ci): clear base-reds on release/v3.8.50 (round 4)
Drains the HARD failures reported by Release-Green run 31693210948 on issue #9985
(ESLint errors: 2) plus the merge-integrity red every open PR is inheriting.
- ESLint error 1: @omniroute/opencode-plugin/src/index.ts had a stray extra
'});' (introduced by #9316) that broke parsing with 'unexpected file in NFT list'
on the build path.
- ESLint error 2: cli-env-inline-comment-10100.test.ts used new Function to extract
parseEnvValue from the bin entrypoint (no-new-func, Hard Rule #3). Extracted the
helper to bin/cli/utils/parseEnvValue.mjs and import it from both the entrypoint
and the test (same behavior, no eval).
- open-sse-typecheck (Fast Quality Gates): open-sse/utils/stream.ts imported
sseCommentsEnabled twice (#9378) causing TS2300 Duplicate identifier; removed the
duplicate import.
- Merge integrity (changelog + generated skills): skills/omni-settings/SKILL.md was
edited manually by #10169 without updating the generator source, so
check:agent-skills-sync failed on every PR (Generated: 1). Moved the curated
thinking-budget content into a <!-- skill:custom-start --> block (the documented
preservation mechanism), which the generator now keeps in sync.
Refs #9985
* fix(tests): align wave1-a poolside test with #10216 probed catalog
#10216 published Poolside's two authenticated-probe models
(poolside/laguna-xs-2.1, poolside/laguna-s-2.1) as static seeds, but the
wave1-a free-tier test still asserted 'no invented static model ids'
(entry.models === []), failing every open PR. Separate poolside from the
empty-models assertion and pin its probed catalog explicitly so a future
catalog change is a deliberate update, not a silent drift.
* fix(pack): register parseEnvValue.mjs in PACK_ARTIFACT_REQUIRED_PATHS
The extract of parseEnvValue to bin/cli/utils/parseEnvValue.mjs added a new
direct import to bin/omniroute.mjs, which pack-artifact-entrypoint-closures
enforces against PACK_ARTIFACT_REQUIRED_PATHS. Register the module so a future
tarball omission fails loudly.
* fix(combo): restore default same-model retry semantics after #10217#10217 wired config.failoverBeforeRetry into the same-model retry guard in
both the priority/auto and round-robin loops, but DEFAULT_COMBO_CONFIG
defaulted the flag to true — flipping same-model retry off for every combo
that never touched the setting, not just the opt-in case. Round-4 bisect
(06f41cda63 vs d2fd88dfbc) reproduced this against
tests/unit/combo-499-abort.test.ts, tests/unit/combo-quota-exhaustion-only-fallback.test.ts
and tests/unit/combo-stream-readiness-fallback.test.ts. Flip the default to
false so the historical retry-before-failover behavior returns for combos
that never set the flag, while explicit opt-in (the two new tests #10217
added to combo-routing-engine.test.ts) still works.
* fix(quality): register visionBridge-responses-9597 in stryker tap.testFiles
check-mutation-test-coverage.mjs flagged tests/unit/guardrails/visionBridge-responses-9597.test.ts
as covering open-sse/services/combo/comboStructure.ts without being listed
in stryker.conf.json's tap.testFiles array. Add it so mutation coverage
attribution stays accurate.
* test(pack): expect parseEnvValue.mjs in the missing-artifact-paths fixture
The prior commit on this branch registered bin/cli/utils/parseEnvValue.mjs
in PACK_ARTIFACT_REQUIRED_PATHS but the "findMissingArtifactPaths flags
missing root runtime files in the tarball" test still hardcoded the old
expected list, so it never accounted for the new required path being
absent from the simulated tarball. Add it in its alphabetical slot.
* chore(lint): prune stale no-explicit-any suppression for call-log-file-rotation
--prune-suppressions found tests/unit/call-log-file-rotation.test.ts no
longer produces the 5 suppressed @typescript-eslint/no-explicit-any
warnings recorded in config/quality/eslint-suppressions.json. Remove the
dead entry so a regression would be caught again. Full-tree run with
--max-warnings 0 is clean: 0 errors, 0 warnings.
* fix(combo): decouple failoverBeforeRetry same-model guard from the skipUpstreamRetry default
Audit found that DEFAULT_COMBO_CONFIG.failoverBeforeRetry has defaulted to
true since before #10217 (predates #2417), and that value also feeds the
independent skipUpstreamRetry mechanism (src/sse/handlers/chat.ts:859,1126).
The previous commit on this branch flipped that default to false to fix the
#10217 same-model retry guard, which silently disabled skipUpstreamRetry's
own default-on behavior for every combo without an opt-in — a regression in
the opposite direction (executor-level retries before the loop's own
failover, changing latency/failure behavior).
Revert the default back to true and decouple the two mechanisms instead:
resolveComboConfig/resolveComboSetupConfig now also compute
failoverBeforeRetryExplicit, true only when a cascade layer (combo/provider/
global) literally sets failoverBeforeRetry to true — not merely inherited
from the default. The #10217 same-model retry guards in combo.ts (priority/
auto and round-robin loops) now read failoverBeforeRetryExplicit instead of
config.failoverBeforeRetry, restoring opt-in-only behavior for that guard
while the skipUpstreamRetry pass-through (config.failoverBeforeRetry at
combo.ts:1297,2865) is untouched and keeps its historical default-on.
* fix(combo,i18n): align getDefaultComboConfig with 10217 explicit flag; pt denoRelay entities
Two round-4 follow-ups exposed by the combinated base-red PR run:
1. comboConfig.ts: #10217 round-4 fix (104afeda4e) added
failoverBeforeRetryExplicit to resolveComboConfig/resolveComboSetupConfig
but getDefaultComboConfig() returned only DEFAULT_COMBO_CONFIG, so the
combo-config.test.ts deepEqual (resolveComboConfig(null) ===
getDefaultComboConfig()) failed on the extra field. Mirror the opt-in flag
as false in the default.
2. pt.json: denoRelayOrgDomainHint still carried raw <app-name>/<org-slug>
(the UNCLOSED_TAG RSC regression) — encode as <...> like the other
42 locales, greening i18n-deno-relay-unclosed-tag.test.ts.
* chore(lint): disable @next/next/no-location-assign-relative-destination pending per-case review (#10292)
The eslint-config-next bump in #10043 shipped this new rule, flagging 6
pre-existing window.location.href navigations — several are deliberate
full-page reloads (login/logout state reset). Off with tracking issue
rather than a blanket router.push rewrite.
* fix(i18n): fill 439 missing UI keys (thinkingMode ×39 locales + pt catch-up) to restore 100% coverage
The #10169 Thinking Budget keys existed only in en/pt-BR/vi and the pt (PT-PT)
catalog from #10250 lagged 88 recent keys, dropping i18nUiCoverage to 99.3%
vs the frozen 100% ratchet baseline. Translated via the i18n:sync-ui marker
pipeline; glossary + ICU placeholder post-pass clean.
* fix(i18n): zh-TW glossary — replace retired 默認 with canonical 預設 in new thinkingMode keys
* chore(quality): rebase dead-code baseline 248 -> 409 for knip 6.32 bump (#10043)
dependabot #10043 upgraded knip 6.27 -> 6.32, which detects 162 MORE
genuinely-unused exports (331 vs 169) that 6.27 missed; DEAD_FILES
unchanged (78). Reproduced identically on the clean release/v3.8.50 tip
266e39d3 with a fresh 6.32 node_modules, so every PR is born red until
the tool change is absorbed. Owner authorized rebaseline (2026-08-13 via
PR #10260). Structural cleanup of the newly-surfaced dead exports remains
separate debt.
---------
Co-authored-by: adevwithpurpose <adevwithpurpose@users.noreply.github.com>
Co-authored-by: backryun <bakryun0718@proton.me>
* fix(deps): bump nanoid, dompurify for 2 new Dependabot alerts (#189, #190)
Bumps: nanoid ^3.3.17 (was transitive, now overridden), dompurify ^3.4.13
(with monaco-editor scoped override). Closes Dependabot #189, #190.
Remaining #182-#188 (js-yaml + mermaid) already closed by #9651 merge —
awaiting Dependabot re-scan.
npm audit → 0 vulnerabilities.
* fix(repo): harden .gitignore to also ignore a _tasks symlink (/_tasks)
_tasks is a SEPARATE nested git repo (gitignored). The pattern _tasks/ (trailing
slash) ignores only a directory, not a SYMLINK named _tasks. A self-referential
_tasks symlink can slip in via git add -A and, once pulled, checkout materializes
it over the real _tasks repo (destroying plans/specs/hands-off). Anchored /_tasks
ignores the symlink too, preventing re-capture.
* fix(combo): make failoverBeforeRetry actually skip the same-model retry
Both same-target retry loops (priority/auto and round-robin) checked
isTransient/maxRetries/providerExhausted but never consulted
config.failoverBeforeRetry, so a rate-limited model still got
maxRetries+1 back-to-back attempts on itself before falling back to a
sibling — the config option (#2417) was only ever wired into
skipUpstreamRetry, a separate lower-level mechanism. Now the same-model
retry is skipped when failoverBeforeRetry is set AND a sibling target
is actually available; with no sibling left, it still retries same-model
since skipping would just burn the last attempt for nothing.
---------
Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@outlook.com>
Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
setLKGP() was only ever called on success — nothing invalidated a "last
known good provider" pin once that provider started failing, so a
*separate* subsequent request kept re-selecting the same just-failed
target via applyStrategyOrdering.ts's LKGP reordering.
Live incident: an OpenClaw request to combo "default" (routerStrategy:
lkgp) got a real reasoning + apply_patch tool call from
opencode-zen/big-pickle, then 3 separate follow-up requests over the
next ~2 minutes each independently re-selected the same big-pickle
target and each timed out with "504 Stream produced no non-ping SSE
event within 95000ms" before the client gave up — instead of failing
over to any of the combo's other 12 models.
Root cause confirmed via code read: circuit breaker and model lockout
deliberately don't react to this failure class (isStreamReadinessFailureErrorBody
exempts STREAM_READINESS_TIMEOUT/combo_target_timeout 504s from tripping
the provider breaker, and REQUEST_SCOPED_UPSTREAM_ERROR_CODES suppresses
model-lockout recording for the same class — both intentional, to avoid
poisoning a healthy provider on request-specific timing). Nothing else
in the system was clearing the stale LKGP pin, so it kept winning
target-selection ordering for every new top-level request.
Fix: add clearLKGP(comboName, modelId) to src/lib/db/settings/lkgp.ts,
export it through settings.ts/localDb.ts, and call it (mirroring the
existing setLKGP-on-success call pattern exactly, same two keys) in both
combo.ts's per-target failure paths -- handleComboChat's "Done retrying
this model" block and handleRoundRobinCombo's structurally identical
twin -- right where a target is finally given up on and the loop moves
to the next one.
TDD: new regression test in tests/unit/combo-routing-engine.test.ts
("clears LKGP after the last-known-good target fails") reproduces the
exact live scenario -- confirmed failing against the pre-fix code,
passing after. Added direct unit coverage for clearLKGP itself in
tests/unit/db-settings-crud.test.ts (deletes only the targeted key,
sibling keys survive; no-op on an unset key doesn't throw) and
registered the new export in db-settings-split.test.ts's public API
surface characterization test.
Test plan:
- Full combo/LKGP-related suite (combo-routing-engine, db-settings-crud,
db-settings-split, combo-strategy-fallbacks,
combo-selected-connection-success,
delete-provider-connection-invalidates-lkgp-8887, db-read-cache) --
183/183 passing.
- npx tsc --noEmit -- clean for all changed files (pre-existing unrelated
errors elsewhere in the same test files confirmed identical against a
pristine upstream/release/v3.8.50 checkout, zero diff at those lines).
- npm run lint -- clean (new test's any usage properly typed, not left
to inflate the file's frozen any-budget suppression).
⚠️ base-red inherited: #9985
* fix(combo): keep queue/network timeouts out of the provider breaker
A single-model network error (ECONNREFUSED / proxy_unreachable) means we never
reached the provider — the provider may be healthy while only the network path
is broken. OmniRoute's own rate-limit queue timeouts are backpressure we
applied, not an upstream failure. Neither should trip the whole-provider
breaker.
- chatPredicates: the single-model path excludes proxy_unreachable and
RATE_LIMIT_QUEUE_* from the provider-breaker trip.
- accountFallback.recordProviderFailure: isQueueTimeout short-circuits before
the breaker ever counts (combo.ts already flags it from errorText).
- chat.ts: the queue/network guard on the allRateLimited _onFailure trip.
Deliberately leaves the combo same-provider dead-proxy leg (#8376) intact:
there a proxy_unreachable on the next same-provider target must still be able
to open the breaker, or a dead proxy burns every attempt until the 503
max-retry limit.
Signed-off-by: Minxi Hou <houminxi@gmail.com>
* fix(resilience): dedup same-provider network errors per event
Same-provider combo targets can all fail the same single network event (a VPN
blip) within one request. Without a dedup each target counts once toward the
provider breaker, so one transient blip opens the whole-provider breaker while
the provider is healthy — the antigravity outage this branch originally chased.
recordProviderFailure now keeps a short per-provider window (10s) for
proxy_unreachable failures: the first network error in a window counts, the rest
of that window are the same event and return. A genuinely dead proxy keeps
failing across requests (past the window) and still accumulates to its
threshold, so the #8376 dead-proxy protection is not weakened.
Covered by tests/unit/breaker-network-error-guard.test.ts: same-window errors
dedup to one, cross-window errors still open the breaker.
Signed-off-by: Minxi Hou <houminxi@gmail.com>
---------
Signed-off-by: Minxi Hou <houminxi@gmail.com>
* Bypass proxy compaction for native Codex context
* Add native ChatGPT Web provider pipeline
* Add managed browser and tunnel deployment
* Add ChatGPT Web setup and doctor UI
* Document and test ChatGPT Web integration
* fix(security): register chatgpt-web-codex-doctor in LOCAL_ONLY_API_PATTERNS
The diagnostic route under /api/providers/{id}/chatgpt-web-codex-doctor
was not registered in the spawn-capable route guard. Adding it for
parity with the existing /login pattern.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(providers): route chatgpt-web-codex admin routes through a service boundary
The provider CRUD/doctor routes imported chatgpt-web-codex helpers
(finalizeValidatedChatGptWebCodexSecrets, encode/decodeChatGptWebCodexSecrets,
getChatGptWebCodexDoctorStatus) directly from open-sse/executors/**, which
no-restricted-imports (EXECUTOR_IMPORT_RESTRICTION) forbids for src/app/**
files — executor implementations must stay behind an open-sse handler or
service boundary.
Add open-sse/services/chatgptWebCodexAdmin.ts as a thin re-export boundary
(mirroring the existing tokenRefresh.ts re-export pattern) and import from
there instead. No behavior change.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Add a per-target priority option that advances only after trusted quota exhaustion while preserving retry, nested Combo, quality, and Global Fallback semantics.
* feat(sse): server-side template expansion for combo system prompts (#5501)
* fix(quality-gates): register combo-system-prompt-templates-5501 test in stryker tap.testFiles
check:mutation-test-coverage --strict flagged tests/unit/combo-system-prompt-templates-5501.test.ts
as covering src/shared/utils/circuitBreaker.ts without being listed in stryker.conf.json
tap.testFiles, so its mutant kills wouldn't count.
Co-authored-by: maxmad64bis <maxmad64bis@users.noreply.github.com>
---------
Co-authored-by: Max <maxmad64@gmail.com>
Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
Co-authored-by: maxmad64bis <maxmad64bis@users.noreply.github.com>
The combo success path called recordProviderSuccess (cooldown-only)
without notifying the circuit breaker. When a provider breaker entered
HALF_OPEN after repeated failures, successful probe requests never
transitioned it back to CLOSED -- the breaker stayed stuck indefinitely.
Production evidence: agy breaker HALF_OPEN with 699 requests at 98%
success rate, never recovering.
Root cause: combo.ts calls recordProviderSuccess from
providerCooldownTracker.ts (resets cooldown failureCount only) but
never calls breaker._onSuccess(). The failure path in accountFallback.ts
calls breaker._onFailure(), creating an asymmetry.
Fix: add recordProviderSuccess to accountFallback.ts as the symmetric
counterpart of recordProviderFailure. Uses getProviderBreaker (not
configureProviderBreaker) to avoid overwriting the breaker's resetTimeout
with default profile values. Calls breaker._onSuccess() for all non-OPEN
states (CLOSED/DEGRADED/HALF_OPEN), matching execute()'s behavior.
* Fix custom tool output pairing during compression (#8932)
* Bypass proxy compaction for native Codex context
* fix(sse): extract Codex tool-call output repair to leaf module for file-size gate
repairMissingCodexToolCallOutputs (added by #8932 for custom_tool_call
pairing) pushed codex.ts past the frozen file-size baseline. Extract it
to open-sse/executors/codex/toolCallRepair.ts, leaving only the wiring
call in codex.ts. Rebaseline the test file's genuine +41 line growth
from #8932's new custom_tool_call_output coverage.
Co-authored-by: JxnLexn <JxnLexn@users.noreply.github.com>
---------
Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
Co-authored-by: JxnLexn <JxnLexn@users.noreply.github.com>
The commit for #9630 introduced tab characters instead of 2-space
indentation in two blocks (handleComboChat and handleRoundRobinCombo).
Tabs in TypeScript cause TS1128 parsing errors because the parser
expects consistent space-based indentation.
Fix: replace all leading tabs with the proper 2-space indentation
level matching the surrounding codebase convention.
This restores typecheck:core to a clean state on the release branch.
* fix(resilience): count STREAM_EARLY_EOF as a provider failure in combo routing
A STREAM_EARLY_EOF is an upstream that accepted the request (HTTP 200), opened
the SSE stream, then closed it without emitting a single non-ping event. The
combo path classified it together with STREAM_READINESS_TIMEOUT through
isStreamReadinessFailureErrorBody(), and the readiness exemption in
shouldRecordProviderBreakerFailure meant the whole-provider circuit breaker
never saw it.
During a provider-wide outage that makes the breaker blind. Over a 7-day window
on our router we recorded 311 of these events, 302 of them on one model, 265
inside the upstream's published incident window — and the provider breaker sat
at CLOSED / failure_count=0 the entire time. Every request kept being dispatched
to the failing provider instead of shedding to the next combo target.
The two codes are different signals. The readiness probe is a pre-flight
liveness check on a connection we have not committed to, so failing it means
"this connection looks stale". An early EOF means the provider took the request
and then failed to serve it. The single-model path already treats it that way:
shouldTripProviderBreakerForResult has no readiness exemption, so a 502 early
EOF trips the breaker there. This makes the combo path consistent.
isStreamReadinessFailureErrorBody keeps matching both codes, because the
transient-retry and round-robin semaphore-cooldown paths in combo.ts do want
identical treatment for both. Only the breaker needs to tell them apart, so the
distinction is added as a narrow predicate and an optional argument rather than
by changing the shared classifier. Omitting the new argument reproduces the
previous behaviour exactly.
Follows the additive-override pattern established by the isProxyUnreachable
work, and leaves the existing exclusions for client aborts and plain 429s
untouched.
* test: register stream-early-eof-breaker in stryker tap.testFiles
The mutation test-coverage gate (check:mutation-test-coverage --strict)
detects unit tests that cover a mutated module but are missing from
stryker.conf.json tap.testFiles, so their mutant kills would not count.
comboPredicates.ts is one of the mutated modules, and the new
stream-early-eof-breaker.test.ts covers it, so the gate correctly flagged
the omission. 8376-econnrefused-breaker.test.ts -- the test this one is
modeled on -- is already registered; this just brings the new file in line.
No production code change.
---------
Co-authored-by: Nick Sullivan <nick@technick.ai>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* refactor(sse): extract combo dispatch prelude into combo/dispatchPrelude.ts
Pure move, no behaviour change. First of ~7 PRs decomposing the combo.ts
god-file (#3501).
handleComboChat evaluates a series of dispatch branches before it ever
reaches target resolution or the sequential attempt loop. None of them
iterate targets in priority order or need the failover/retry/credential
gate machinery that follows, so they move to a leaf:
- context-cache pin routing (Fix#679), including the
pinIsDurablyUnhealthy / isPinnedModelDurablyUnhealthy health gate
- fusion panel dispatch + the #6455 misconfiguration warn
- pipeline chaining
- nested combo-ref execute-mode runtime-unit dispatch
Only the chaos and round-robin hand-offs stay inline (11 and 13 lines);
extracting those would be pure indirection.
open-sse/services/combo.ts 3642 -> 3341 (-301)
open-sse/services/combo/dispatchPrelude.ts: 619 (under the 800 cap)
Each helper keeps the fall-through protocol the inline blocks had: return
a Response to OWN the request, return null to fall through. A flipped
null/Response would silently bypass the whole combo strategy, so the new
tests pin both directions for every branch.
combo.ts re-exports pinIsDurablyUnhealthy so combo-pin-health-gate.test.ts
keeps resolving. The leaf takes handleComboChat as a `runCombo` parameter
instead of importing it, so combo/ keeps zero back-edges into combo.ts.
Complexity-neutral: the first cut added +3 violations (two
max-lines-per-function, one complexity) inside the new leaf, so
evaluatePinnedResponse, orderRuntimeUnits, recordRuntimeUnitStickySuccess
and buildBaseOptions were split out. check:complexity now measures 2169
and check:cognitive-complexity 956 — identical to the pristine base.
* test(sse): close the dispatch-prelude coverage holes found by mutation testing
An adversarial mutation audit of the suite added in the previous commit
found it guarded the fall-through protocol well but asserted almost
nothing about what the helpers do once they OWN the request. 5 of 12
seeded mutations survived. Worst case: deleting the pinned-model
dispatch call outright left all 12 tests green.
Three holes, now closed (8 tests -> 20):
Hole A — the honored-pin path had zero coverage. Both existing pin tests
DROP the pin, so the dispatch, the 200-but-empty quality gate, the
[408, 429, 500, 502, 503, 504] failover list and the catch(pinErr)
branch were unguarded — exactly the logic the 2026-06-21 / 2026-06-22
incident comments call load-bearing. Adds five tests over a seeded
healthy provider connection so the pin is actually honored.
Hole B — orderRuntimeUnits was only ever driven with `priority`, which
is a no-op through it. Four of five strategy branches could be deleted
with nothing failing. Adds round-robin rotation and weighted sticky
ordering tests.
Hole C — recordRuntimeUnitStickySuccess never did anything under test:
both its guards need weighted/round-robin, so an early return changed
nothing. Covered by the new sticky-batch test.
Verified by re-running the mutations rather than assuming: all 7 that
previously survived (delete-pin-dispatch, serve-despite-failed-quality,
never-fail-over-on-transient, rr-counter-not-advanced, rotation-removed,
weighted-sticky-skipped, sticky-recording-no-op) are now killed.
The first sticky-batch test I wrote was itself vacuous — asserting "same
unit twice" holds equally when the recording helper is stubbed out, since
nothing advances the counter either. It now asserts the batch runs out
and rotation resumes on the third dispatch, which is what actually
distinguishes the two.
Also restores API_KEY_SECRET in test.after; it was set at module load and
never put back, inconsistent with the DATA_DIR handling beside it.
* fix(ci): teach known-symbols gate the relocated fusion/pipeline dispatch
The combo sub-check of check:known-symbols asserts every canonical routing
strategy has a real dispatch branch. It scanned a hardcoded file list and
matched only `strategy === "..."`, so the prelude extraction tripped it twice:
[combo] 2 estratégia(s) canônica(s) sem branch de despacho em combo.ts:
✗ fusion
✗ pipeline
Both branches are still wired — they just moved to combo/dispatchPrelude.ts and
took the early-return guard form `if (strategy !== "fusion") return null;` that
extracting a branch into a `tryXDispatch()` leaf naturally produces.
Two changes, both extending existing precedent (the list already carries the
Block J leaves for the same reason):
- register combo/dispatchPrelude.ts in comboDispatchFiles
- widen the extractor to `strategy [!=]== "..."` so the inverted guard counts
Loose `==`/`!=` stay rejected, and no `handledNotCanonical` fallout: the gate
now reports 20 canonical strategies, all 20 via despacho.
* chore(ci): register combo-dispatch-prelude test in stryker tap.testFiles
check:mutation-test-coverage --strict failed once the known-symbols fix let
Fast Quality Gates advance to it:
✗ 2 covering unit test(s) across 2 module(s) are missing from
stryker.conf.json tap.testFiles
open-sse/services/combo/comboStructure.ts
open-sse/services/combo/rrState.ts
The new tests/unit/combo-dispatch-prelude.test.ts exercises both modules, and
both are already in stryker's mutate list, so without the registration its
mutant kills would not have counted toward the nightly mutation gate.
Note (unchanged, still out of scope): combo/dispatchPrelude.ts itself is not in
stryker's `mutate` list. Adding it would widen the nightly mutation surface,
which is a separate call from fixing this drift.
* docs(changelog): add fragment for #8582 combo dispatch prelude
* refactor(sse): extract combo target resolution into combo/targetResolution.ts
Pure move, no behaviour change. Lifts the target-resolution stage of
handleComboChat — everything between the dispatch prelude and the attempt
loop — into a new leaf, open-sse/services/combo/targetResolution.ts.
Moved verbatim: provider-wildcard expansion, weighted step-group resolution
+ sticky-weighted eligibility, request-tag routing, the known-context-overflow
early return, the smart/pipeline-enabled auto dispatch, auto-strategy
ordering, per-strategy ordering, cache-strategy affinity, session stickiness,
eval scores, request-compatibility + context-requirement filters, task-aware
reordering, prompt-cache affinity, and the priority-strategy pre-screen.
The three early exits become an { earlyResponse } result so the host decides
to return them (same pattern as resolveAutoStrategyOrder). The values the
attempt loop still reads — orderedTargets, stickyWeightedLimit,
getWeightedStepKeyForTarget, the session-stickiness result and preScreenMap —
are returned instead of closed over. Loop config (maxRetries, retryDelayMs,
fallbackDelayMs, maxSetRetries, setRetryDelayMs) stays in combo.ts.
buildAutoCandidates is dependency-injected because it lives in combo.ts, so
the leaf keeps zero back-edges into its host.
combo.ts 3640 -> 3321 lines; new leaf 484 lines (under the 800 cap).
Part of the #3501 god-file decomposition campaign.
* refactor(sse): split targetResolution into stage helpers, ratchet combo.ts file-size baseline
Follow-up to the target-resolution extraction: the moved region landed as one
311-line function, which converted inline code inside the (already-violating)
handleComboChat into a NEW separately-counted violating function — check:complexity
2169 -> 2171 and check:cognitive-complexity 956 -> 957.
Split resolveComboTargetPipeline along its natural stage boundaries into 14
helpers (wildcard expansion, weighted eviction/eligibility/sticky-key/selection,
step-key mapper, context-overflow response, pool-size log, smart-pipeline dispatch
and its fall-through logger, strategy ordering, continuity filters, task-aware
ordering, prompt-cache enablement/first-target protection/affinity stage). Each
stage takes the previous stage's output and returns the next; still a pure move.
The leaf now contributes ZERO complexity, max-lines-per-function and
cognitive-complexity violations. Both ratchets are back at base 4053e2314 values:
check:complexity 2169, check:cognitive-complexity 956. (Both still print RED
against their frozen ceilings 2130/951 — pre-existing base-red per #8580.)
Also ratchets ONLY the open-sse/services/combo.ts entry in
config/quality/file-size-baseline.json from 3642 to 3322, with a justification
note in the file's existing style. No sweep of unrelated entries.
* chore: stack targetResolution on dispatchPrelude tip, rebank + skills
Rebased onto refactor/combo-dispatch-prelude. Keep both leaves in
check-known-symbols. Regenerate file-size baseline; sync agent skills.
* fix(sse): restore #8494 capability fail-closed after targetResolution extract
Stacking targetResolution onto the dispatchPrelude tip dropped the #8488/#8494
compatFilterFailOpen wiring: hard capability filters emptied the pool into a
generic 404 no_executable_targets, and fail-open never re-admitted the pool.
Restore describeCapabilityFilterExhaustion earlyResponse in
applyContinuityFilters and the matching round-robin path, then rebank the
file-size baseline for tip growth the incomplete prior rebank missed.
* fix(sse): realign model-lockout cooldown options with the post-#8254 type
This branch predates #8254, which renamed the recordModelLockoutFailure option
`exactCooldownVerified` -> `exactCooldownIsUpstreamReset` and changed combo.ts's
predicate from `lockoutHintVerified` (#8393's `lockoutHintMs > 0`) to
`lockoutHintMs > mlSettings.baseCooldownMs`. Rebasing onto the current tip brought
the renamed type without updating these two call sites, so typecheck:core failed
with TS2353 at both.
Restores the base expression verbatim rather than re-wiring `lockoutHintVerified`
under the new name. The base predicate is the correct one: selectLockoutCooldownMs
returns the parsed hint ONLY when `lockoutHintMs > baseCooldownMs`, and otherwise
returns 0 or a synthetic baseCooldownMs — so `lockoutHintMs > 0` would mark a
synthetic cooldown as an upstream reset and let it bypass the #7940 maxCooldownMs
cap, which is the bug #8254 fixed.
---------
Co-authored-by: MumuTW <johnsxn.us@gmail.com>
Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* refactor(sse): extract combo dispatch prelude into combo/dispatchPrelude.ts
Pure move, no behaviour change. First of ~7 PRs decomposing the combo.ts
god-file (#3501).
handleComboChat evaluates a series of dispatch branches before it ever
reaches target resolution or the sequential attempt loop. None of them
iterate targets in priority order or need the failover/retry/credential
gate machinery that follows, so they move to a leaf:
- context-cache pin routing (Fix#679), including the
pinIsDurablyUnhealthy / isPinnedModelDurablyUnhealthy health gate
- fusion panel dispatch + the #6455 misconfiguration warn
- pipeline chaining
- nested combo-ref execute-mode runtime-unit dispatch
Only the chaos and round-robin hand-offs stay inline (11 and 13 lines);
extracting those would be pure indirection.
open-sse/services/combo.ts 3642 -> 3341 (-301)
open-sse/services/combo/dispatchPrelude.ts: 619 (under the 800 cap)
Each helper keeps the fall-through protocol the inline blocks had: return
a Response to OWN the request, return null to fall through. A flipped
null/Response would silently bypass the whole combo strategy, so the new
tests pin both directions for every branch.
combo.ts re-exports pinIsDurablyUnhealthy so combo-pin-health-gate.test.ts
keeps resolving. The leaf takes handleComboChat as a `runCombo` parameter
instead of importing it, so combo/ keeps zero back-edges into combo.ts.
Complexity-neutral: the first cut added +3 violations (two
max-lines-per-function, one complexity) inside the new leaf, so
evaluatePinnedResponse, orderRuntimeUnits, recordRuntimeUnitStickySuccess
and buildBaseOptions were split out. check:complexity now measures 2169
and check:cognitive-complexity 956 — identical to the pristine base.
* test(sse): close the dispatch-prelude coverage holes found by mutation testing
An adversarial mutation audit of the suite added in the previous commit
found it guarded the fall-through protocol well but asserted almost
nothing about what the helpers do once they OWN the request. 5 of 12
seeded mutations survived. Worst case: deleting the pinned-model
dispatch call outright left all 12 tests green.
Three holes, now closed (8 tests -> 20):
Hole A — the honored-pin path had zero coverage. Both existing pin tests
DROP the pin, so the dispatch, the 200-but-empty quality gate, the
[408, 429, 500, 502, 503, 504] failover list and the catch(pinErr)
branch were unguarded — exactly the logic the 2026-06-21 / 2026-06-22
incident comments call load-bearing. Adds five tests over a seeded
healthy provider connection so the pin is actually honored.
Hole B — orderRuntimeUnits was only ever driven with `priority`, which
is a no-op through it. Four of five strategy branches could be deleted
with nothing failing. Adds round-robin rotation and weighted sticky
ordering tests.
Hole C — recordRuntimeUnitStickySuccess never did anything under test:
both its guards need weighted/round-robin, so an early return changed
nothing. Covered by the new sticky-batch test.
Verified by re-running the mutations rather than assuming: all 7 that
previously survived (delete-pin-dispatch, serve-despite-failed-quality,
never-fail-over-on-transient, rr-counter-not-advanced, rotation-removed,
weighted-sticky-skipped, sticky-recording-no-op) are now killed.
The first sticky-batch test I wrote was itself vacuous — asserting "same
unit twice" holds equally when the recording helper is stubbed out, since
nothing advances the counter either. It now asserts the batch runs out
and rotation resumes on the third dispatch, which is what actually
distinguishes the two.
Also restores API_KEY_SECRET in test.after; it was set at module load and
never put back, inconsistent with the DATA_DIR handling beside it.
* fix(ci): teach known-symbols gate the relocated fusion/pipeline dispatch
The combo sub-check of check:known-symbols asserts every canonical routing
strategy has a real dispatch branch. It scanned a hardcoded file list and
matched only `strategy === "..."`, so the prelude extraction tripped it twice:
[combo] 2 estratégia(s) canônica(s) sem branch de despacho em combo.ts:
✗ fusion
✗ pipeline
Both branches are still wired — they just moved to combo/dispatchPrelude.ts and
took the early-return guard form `if (strategy !== "fusion") return null;` that
extracting a branch into a `tryXDispatch()` leaf naturally produces.
Two changes, both extending existing precedent (the list already carries the
Block J leaves for the same reason):
- register combo/dispatchPrelude.ts in comboDispatchFiles
- widen the extractor to `strategy [!=]== "..."` so the inverted guard counts
Loose `==`/`!=` stay rejected, and no `handledNotCanonical` fallout: the gate
now reports 20 canonical strategies, all 20 via despacho.
* chore(ci): register combo-dispatch-prelude test in stryker tap.testFiles
check:mutation-test-coverage --strict failed once the known-symbols fix let
Fast Quality Gates advance to it:
✗ 2 covering unit test(s) across 2 module(s) are missing from
stryker.conf.json tap.testFiles
open-sse/services/combo/comboStructure.ts
open-sse/services/combo/rrState.ts
The new tests/unit/combo-dispatch-prelude.test.ts exercises both modules, and
both are already in stryker's mutate list, so without the registration its
mutant kills would not have counted toward the nightly mutation gate.
Note (unchanged, still out of scope): combo/dispatchPrelude.ts itself is not in
stryker's `mutate` list. Adding it would widen the nightly mutation surface,
which is a separate call from fixing this drift.
* docs(changelog): add fragment for #8582 combo dispatch prelude
* fix(backend): fail closed when capability filters empty the combo pool
Tools/vision/structured_output filters no longer re-admit the full pool when
every target is incompatible. Opt-in via combo config compatFilterFailOpen.
Closes#8488
* fix(backend): keep tool-emulation providers under fail-closed filters
Carve out providers with toolCalling:"emulated" (#5240) from tools
capability_mismatch so chatgpt-web combos still reach the prompt shim.
Align round-robin compatFilterFailOpen with settings fallback and drop
new any-typed params from the #8488 combo-routing tests.
* chore(lint): prune stale combo-routing-engine any suppressions
Test cleanup in #8488 dropped four no-explicit-any hits; sync the freeze file.
* chore(quality): rebaseline combo.ts + freeze new-above-cap comboStructure.ts
check:file-size was red for this PR's own growth: combo.ts grew 3640->3693
(+53, the capability-filter fail-closed guard + compatFilterFailOpen escape
hatch at both call sites) and combo/comboStructure.ts crossed the 800-line
new-file cap at 918 (describeCapabilityFilterExhaustion +
providerSupportsEmulatedToolCalling for the #5240 emulated-tool-calling
exemption). Both are irreducible orchestration wiring at the existing combo
filter chokepoint (same precedent as #7301's cooldown-retry generalization).
Companion test tests/unit/combo-routing-engine.test.ts frozen at its own
grown size (3409->3449). No logic change; 95/95 tests pass.
Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
---------
Co-authored-by: ikelvingo <im.kelvinwong@gmail.com>
Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
Co-authored-by: Prudhvivuda <Prudhvivuda@users.noreply.github.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* test(tail): retire stale i18n __MISSING__ repro + fix qianfan website URL
Base-red slice 6, rebased onto the advanced release/v3.8.49 (91fd5f9). The oauth
grok-cli #7610 guard was already fixed on the base by #8027 (it reads the warning
from grokCliAuthJson.ts) — dropped from this slice to avoid a conflicting duplicate.
Remaining two, still red on the current base:
- i18n #7258: the "focused repro" asserted zh-TW.json STILL carries raw __MISSING__:
placeholders. That backlog was filled (the "no locale has a raw __MISSING__: leaf"
invariant is the durable guard); retired the now-inverted repro.
- qianfan: Baidu renamed the product page (product/wenxinworkshop -> product-s/
qianfan_home); updated the expected website URL.
Validated (clean env): i18n 4/0, qianfan 5/0; oauth-modal-grok 2/0 already green on base.
* fix(resilience): short-circuit combo on input-bound failures (context_length_exceeded) (#8375)
isInputBoundRequestFailure() predicate detects deterministic input-bound
errors (context_length_exceeded/context_window_exceeded). The combo loop
propagates the original 400 immediately instead of burning MAX_GLOBAL_ATTEMPTS
retrying identical oversized inputs against every account.
Test: combo-input-bound-failure-8375.test.ts (1 test, 2 assertions)
* fix(resilience): add early-exit in combo dispatcher for input-bound failures (#8375)
When isInputBoundRequestFailure detects context_length_exceeded,
the combo loop returns {ok:false, response} immediately instead of
re-dispatching the oversized request.
Test: node --import tsx/esm --test tests/unit/combo-input-bound-failure-8375.test.ts
- 1 test, 2 assertions, 0 fail
* fix(translator): strip input_image from tool outputs in Responses->Chat downgrade (#8459)
toolOutputContentToString() extracts input_text/output_text parts and
replaces input_image with a placeholder instead of JSON.stringify'ing
the content-part array (which embedded raw ~52KB base64 as inert text).
Applied to both function_call_output and custom_tool_call_output branches.
Existing translator tests: 88/88 pass.
New tests: 4/4 pass.
* fix(providers): set qwen-web toolCalling to false — web-cookie provider has no native function calling (#8437)
qwen-web is a web-cookie provider that emulates tools via synthetic system
prompt text and <tool> XML parsing, never sending a native tools[] field
upstream. The filterTargetsByRequestCompatibility gate filters out non-tool-
calling targets when the request carries tools, but qwen-web's registry entry
had toolCalling=true, so the filter let it through and a tool-using session
failing over to qwen-web would silently degrade to text-only chat with
'Tool X does not exists' errors.
Sibling web-cookie providers (chatgpt-web, yuanbao-web, claude-web, etc.)
all correctly set toolCalling: false — qwen-web was an outlier introduced
in PR #7874.
Verification:
- LSP diagnostics: clean
- Pattern matches chatgpt-web, yuanbao-web, and other web-cookie providers
* fix(backend): empty upstream response mislabeled as exhausted_connection (#8397)
isEmptyContentFailure guard only matched '/empty content/i' but the actual
error text from detectMalformedNonStream is 'returned an empty response
(no usable choices/output)' — which lacks the word 'content'. Expanded
regex to also match '/empty response/i' so these transient upstream glitches
don't get classified as connection-level exhaustion in combo diagnostics.
Test: 28 existing combo-target-exhaustion tests pass (no new test needed)
* test(#8397): add regression test for empty-response 502 not marking provider/connection exhausted
* test(qwen-web): align registry snapshot with toolCalling:false
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(resilience): scope #8375 input-bound short-circuit to homogeneous remainders
The isInputBoundFailure short-circuit (context_length_exceeded /
context_window_exceeded) fired unconditionally on the first target, aborting
the whole combo even when later targets are a different model with a larger
context window — regressing the intentional heterogeneous-combo fallback that
isContextOverflow400 (#6637) protects. Reproduced with a 2-target combo
(small-context model fails, larger-context model would have succeeded): the
combo never reached target 2.
Scope the short-circuit to remainders where every remaining target shares the
same modelStr as the one that just failed — the "retrying will fail
identically" premise for context_length_exceeded only holds within a
homogeneous same-model pool.
Rebaselines open-sse/services/combo.ts's frozen file-size cap (3642->3679)
for this PR's own combo.ts growth (config/quality/file-size-baseline.json).
Refs #8375
Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
---------
Co-authored-by: Probe Test <probe@example.com>
Co-authored-by: herjarsa <herjarsa@users.noreply.github.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Co-authored-by: ikelvingo <im.kelvinwong@gmail.com>
Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
Two changes with one root cause: several hot paths built a full JSON string only to read
its .length, and one of them silently changed the answer.
1. CORRECTNESS -- combo's fallback-compression trigger
estimateTokens(JSON.stringify(attemptBody)) took the STRING branch of estimateTokens,
which is ceil(length / CHARS_PER_TOKEN) over the raw JSON. An inline base64 image is
then charged as if every character of the data URL were prose. Measured on a 200 KB
inline image:
via string (before) 50,039 tokens
via object (after) 1,231 tokens
a 40x over-count, tripping fallback compression on requests nowhere near the context
window. This is the same class #8368/#8401 fixed on the request path; the combo call
site was missed. Passing the object routes through extractImageTokens, which charges
images structurally. Text-only bodies are unaffected -- verified identical, and pinned
by a test.
2. ALLOCATION -- jsonLength()
Adds an exact serialized-length walker: same O(n) scan, no string. Used by
estimateTokens' object branch and by streamReadinessPolicy (which runs on every
streaming request and only ever used .length).
Exactness matters because every consumer feeds a threshold, so this is property-tested
against JSON.stringify over 4000 generated structures covering escaping, lone
surrogates, omitted values, non-finite numbers, toJSON, Date, Map, cycles and BigInt.
Anything outside the plain-JSON subset falls back to JSON.stringify for THAT SUBTREE
only, so an exotic leaf never forces the message history back onto the allocating path.
Honest scoping of the memory win: the string was always transient, and V8 collects it
efficiently, so this is not 3 MiB of retained heap. Measured allocation churn over 20 calls
on a 3.06 MiB body: 3.1 MiB -> 0.5 MiB, about 6x less. The #8549 benchmark row for this
mechanism measures a HELD string and therefore overstates it; the correctness fix above is
the larger deliverable here.
* fix(sse): shallow per-target copy for the combo attempt body (#7847)
combo.ts deep-cloned the request body for every target. On a 3.05 MiB agent request that
is 9.53 MiB at 3 targets, and it scales linearly:
3 targets 9.53 MiB (3.12x wire) -> ~0.001 MiB
5 targets 15.89 MiB (5.19x wire) -> ~0.000 MiB
10 targets 31.78 MiB (10.39x wire) -> ~0.001 MiB
The isolation it bought only ever needed to contain TOP-LEVEL SCALAR writes. The full
mutation surface on this path is two assignments:
combo.ts bodyRecord.max_tokens = ... (reasoning buffer)
chatCore.ts body.model = model (Background Task Redirection T41)
Nothing mutates the nested payload; applyCompression and injectUniversalHandoffBody both
return new objects (verified empirically -- neither touches its input, and both tolerate a
frozen one). So a fresh top-level object per target gives identical isolation while sharing
the expensive messages/tools arrays.
Also fixes a REAL cross-target leak in handleRoundRobinCombo. It already used a shallow
copy, but took it only when the reasoning buffer actually changed max_tokens -- every other
attempt shared the caller's object outright. The new test reproduces it on the unmodified
code: target 2 received model "mutated-by-openai/gpt-4o-mini". In production that is a
Background Task Redirection on one round-robin target rewriting body.model for the next.
The copy is now unconditional.
The invariant is pinned by tests rather than by a comment listing mutation sites, so the
clone strategy can change again without anyone re-deriving them by hand:
- a target's in-place write must not leak into the next (priority / fill-first /
round-robin; the stub reproduces chatCore's body.model write)
- the caller's body is never mutated
- the per-target copy stays shallow (targets share one messages array)
- freeze probe: combo's own body handling performs no in-place writes
* test(sse): register the combo attempt-body isolation test in stryker tap.testFiles
The new test resets the circuit breaker in beforeEach, so it counts as a covering test
for src/shared/utils/circuitBreaker.ts. Without registering it,
check:mutation-test-coverage --strict reported a 4th drift entry that was not there on the
pristine tip -- new drift introduced by this PR. Registered in sorted position; the gate is
back to the 3 pre-existing entries (accountFallback.ts, error.ts, comboPredicates.ts) that
#8538 addresses.
* fix(resilience): honor comboCooldownWait for every combo strategy
The cooldown-wait path claimed all strategies, but isComboCooldownWaitEligible still gated on quota-share/auto. Widen the gate, raise the per-target timeout floor with it, and consult the allow-list from earliestRetryAfter so a later 403 cannot skip the decision. Fixes#8541.
* test(combo): fold cooldown-wait strategy assertions into a loop
Keep combo-config.test.ts under the file-size ceiling while covering every
routing strategy (including internal quota-share) for the #8541 gate fix.
* chore(combo): trim cooldown-wait comment under file-size ceiling
After rebase onto tip the net +2 in combo.ts sat one line over the frozen
check:file-size count (split-based); fold the SECURITY note into the block.
handleComboChat/handleRoundRobinCombo tracked lastStatus (first-write-wins),
lastError (last-write-wins), and earliestRetryAfter (global MIN across all
targets) independently, so the final unavailableResponse() could surface a
status/message pair from two different failing targets and decorate a
config-class error (e.g. Antigravity's 422 missing_project_id, which carries
no retryAfter of its own) with an unrelated target's long reset window.
- lastStatus now overwrites on every failure (last-write-wins), matching
lastError, so status and message always come from the same target.
- the "(reset after ...)" decoration is only applied when the surfaced
status is itself rate-limit-class (429/503) — see the new
open-sse/services/combo/unavailableRetryGate.ts leaf module (both
combo.ts and chat.ts are already over their file-size baseline, so the
gate logic lives in a new module and combo.ts only wires it in).
Co-authored-by: ikelvingo <im.kelvinwong@gmail.com>
