* fix(codex): fail the stream when the websocket closes before a terminal event
* docs(changelog): add fragment for Codex websocket premature-close fix
* chore(quality): rebaseline file-size baselines for #12737 test and executor growth
* fix(codex): log websocket failures and harden premature-close tests
Review follow-up: failController now logs the failure (code + message)
via nextInput.log, and onclose surfaces the WS close code/reason in the
log line (the public payload stays sanitized through the allowlist).
Adds regression tests for the onerror-before-onclose sequence and a
close with zero prior events.
* chore(quality): re-measure the #12737 codex.ts file-size rebaseline after the release merge
The PR's own annotation was written against base 1505 and the frozen cap was
already at 1528 on release/v3.8.51 (which grew the file independently). With
this PR's +13 lines the merged file measures 1530, so the frozen entry moves
1528->1530 (+2 of genuine PR growth; the remaining 11 lines fit the headroom
the tip already had). No other frozen entry touched.
---------
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(providers): read the vLLM context window from max_model_len
normalizeDiscoveredModels resolved the window from inputTokenLimit,
context_length, contextLength and top_provider.context_length. vLLM
reports it as max_model_len and nothing else, so a synced vLLM model
carried no inputTokenLimit and the resolver fell back to the 128K
default - half the window on a 250K deployment.
The native vllm provider and the OpenAI/Anthropic-compatible custom
providers all pass raw records through this function, so one chain entry
covers the three connection shapes.
Closes#12858
* docs(changelog): fragment for #12897
---------
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* test(dashboard): reactivate request logger coverage
* test(dashboard): assert the request-logger modal by its own label
The drift this PR unblocks is real — the next-intl mock lost its `useLocale`
export in #7935, so the suite died on `No "useLocale" export is defined`.
Restoring it is the actual fix.
The dialog assertions were also loosened to `[role="dialog"]`, and that part
was not needed: 20 components under src/ render that role, so the selector
stops proving this particular modal is the one on screen. Measured — keeping
only the `useLocale` fix and restoring a precise selector still passes 6/6.
Now asserting `[aria-label="ariaLabel"]`, which is what RequestLoggerDetail.tsx:469
renders (the mock returns the key rather than the translation).
---------
Co-authored-by: Paco Cartones <pacocartones@users.noreply.github.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* build/mcp: externalize @modelcontextprotocol/sdk in standalone server bundle
The SDK client graph contains a module-level class-extends-Client cycle
against the top-level-await Client module. Webpack's TLA runtime evaluates
that circular subgraph out of order when it is inlined into route chunks,
throwing "Cannot access 'l' before initialization" during module
evaluation. Every request to /api/mcp/stream then answers HTTP 500 on
initialize and the failed module is evicted and re-evaluated per request,
which floods the logs with the same ReferenceError. Node's native ESM
loader resolves the same circular graph through live bindings, so keep
the SDK external to the server bundle like the other packages that break
only when bundled.
Signed-off-by: Minxi Hou <houminxi@gmail.com>
* changelog: record @modelcontextprotocol/sdk externalize fix (#13859)
Signed-off-by: Minxi Hou <houminxi@gmail.com>
---------
Signed-off-by: Minxi Hou <houminxi@gmail.com>
* fix(sse): strip Codex temperature on native Responses passthrough
Codex /responses rejects sampling params with FastAPI 400
Unsupported parameter: temperature. Native passthrough returned
before the Responses allowlist, so client temperature reached
upstream on combo traffic.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(sse): extract Codex passthrough param strip under file-size cap
Keep temperature/top_p (and #3317 client-only fields) stripped before
native Codex /responses passthrough returns. Move the call to
open-sse/executors/codex/stripPassthroughRejectedParams.ts so
executors/codex.ts stays under its frozen 1505-line cap.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(oauth): classify an embedded invalid_grant in a refresh error body
Cline answers a dead refresh_token with
400 {"data":"","error":"failed to refresh token: invalid_grant","success":false}
The code is the tail of a sentence — neither a bare code nor an
"error":"<code>" field pair — so extractOAuthErrorCode returned null.
A null classification means refreshClineToken emits no unrecoverable
sentinel, so a permanently consumed refresh_token is handled as a
TRANSIENT failure. tokenHealthCheck therefore never reaches its
unrecoverable branch, and never runs the credentialsChangedSinceSweep
race guard, the "please re-authenticate this account" message, or the
dead-token clear for rotating providers. The connection instead stays
active with errorCode "refresh_failed", retries the same consumed token
3x per sweep behind an exponential backoff, and 401s every request
routed to it indefinitely with no actionable operator signal.
Scan for a known unrecoverable code embedded in the error value as a
last resort, after the exact-match and nested-JSON paths, delimited on
both sides so server_error, xinvalid_grant, my_invalid_grant_flag and a
502 HTML page all still classify as null.
Also add cline to ROTATION_LOCK_GROUP: refreshClineToken reads a new
refreshToken out of every response body and a measured refresh rotated a
connection's stored token, so sibling connections must not refresh
concurrently. cline was already listed in tokenHealthCheck's
ROTATING_REFRESH_PROVIDERS but missing from the serializer.
* docs(changelog): add fragment for cline refresh token error classification
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(api): accept blockedModels in the key permissions schema
`PATCH /api/keys/[id]` already destructures `blockedModels`, forwards it
into the update payload, and `updateApiKeyPermissions()` writes it to the
`blocked_models` column. Only the first link was missing:
`updateKeyPermissionsSchema` never declared the field, so Zod stripped it
from the parsed body and the destructured value was always `undefined`.
The request answered 200 and wrote nothing.
The API Manager permissions modal sends `blockedModels` on every save
(ApiManagerPageClient.tsx), so the Claude-Code family-blocking control
silently did nothing and an existing deny-list could not be cleared.
`blockedModels` is the deny-list half of the model policy — read by
`isModelAllowedForKey()` before the allow-list and winning over it — so
that half was only reachable by editing the database by hand.
Declare the field mirroring `allowedModels` (trimmed, non-empty, max
1000) and count it in the "No valid fields to update" guard so a body
carrying only `blockedModels` is a valid update.
Left out of `createKeySchema` deliberately: the create route does not
read `blockedModels`, so declaring it there would be dead weight.
* docs(changelog): add fragment for blockedModels key schema fix
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(sse): prevent Anthropic 400s for Claude-native handoffs
* docs(changelog): add fragment for Claude-native handoff 400 fix
* refactor(sse): satisfy file-size and complexity ratchets
Keep the Claude wire-body guard while staying under the frozen per-file line baselines and the complexity ratchets measured against release/v3.8.51.
Extract the final constraint coordinator, split system-message normalization into focused helpers, and isolate handoff response parsing. Reflow the universal-handoff explanation to absorb the added source-format argument without growing the frozen file.
* fix(sse): Anthropic OAuth 403 "Request not allowed" is a per-request refusal, not a ban
A single upstream 403 on the `claude` OAuth connection was classified
FORBIDDEN and written as the terminal `banned` connection state
(chatCore -> writeTerminalStatus). From then on every request to that
provider was short-circuited with "All 1 connection(s) banned by
upstream - please reconnect in the dashboard" without touching Anthropic,
until an operator reconnected.
Anthropic's OAuth surface answers a small fraction of otherwise-valid
requests with 403 {"type":"permission_error","message":"Request not
allowed"}. On the reporting install the same token returned 200 forty
seconds before the 403 and again right after the connection was
re-enabled; a revoked or expired token is a 401 authentication_error, not
this. It is a refusal of one request, not of the credential.
Classify it as the new non-terminal PROVIDER_ERROR_TYPES.REQUEST_REJECTED
(scoped to provider `claude` and the "Request not allowed" body) and list
that type in authTerminalStatus.isNonTerminalProviderError, mirroring the
Cloudflare FINGERPRINT_REJECTION precedent. The combo layer still falls
through to the next target for the failing request; the connection stays
active for the next one. Any other claude 403 keeps its previous
classification.
Tests: error-classifier.test.ts covers the Anthropic body, the
gateway-flattened "[403]: Request not allowed" message, the same body from
a non-Anthropic provider (still FORBIDDEN), other claude 403s (unchanged),
and the helpers; anthropic-request-not-allowed-not-a-ban.test.ts pins
resolveTerminalConnectionStatus() -> null for the new type even with a
`permanent` fallback verdict, and `banned` for a generic claude 403.
* fix(sse): cooldown with backoff and streak escalation for REQUEST_REJECTED (#12859)
Not "ignore the 403" either: if Anthropic ever made "Request not allowed"
systematic, re-sending every request into it would be the wrong thing to
do to an OAuth account. chatCore now handles REQUEST_REJECTED explicitly:
- exclude the connection via setConnectionRateLimitUntil for a growing
cooldown (5 -> 15 -> 45 min) so a sporadic refusal costs minutes, not a
reconnect, and a systematic one cannot become a stream of 403s;
- escalate to the terminal `banned` state only for 3 refusals within a
60-minute window (services/requestRejectedStreak.ts, in-memory per
connection; a restart forgets the streak, erring towards more cooldowns
rather than an operator-undone ban), with a last_error that says so;
- probe-origin failures record but never cool down or ban (#9817).
The existing "request not allowed" text rule (5 s) is unaffected:
markAccountUnavailable skips a connection that already has a future
rateLimitedUntil, so the minute-scale cooldown written here wins.
Tests: request-rejected-streak.test.ts pins the window/threshold/backoff
arithmetic; anthropic-request-not-allowed-cooldown-escalation.test.ts drives
the real chat route against a mocked 403 upstream on a `claude` OAuth
connection: 300 s cooldown, then 900 s, then banned on the third refusal;
a different claude 403 body still bans on the first response.
* chore(changelog): name the #12859 fragment after its PR (#12864)
* refactor(sse): move the REQUEST_REJECTED branch into a chatCore leaf; register its tests for mutation coverage
chatCore.ts is frozen at 5984 lines by the file-size ratchet; the branch
body now lives in open-sse/handlers/chatCore/requestRejectedFailure.ts
(chatCore: 5974 -> 5983). stryker.conf.json tap.testFiles gains the two new
DB-backed tests so their mutant kills count (check:mutation-test-coverage).
* fix(sse): count refusal episodes, reset on success, keep the dashboard honest (#12859 review)
Review findings on the first cut of the REQUEST_REJECTED handling:
- A burst of in-flight requests that all got the 403 within seconds
produced streak 1, 2, 3 and a ban from one upstream event. The streak
now counts cooldown *episodes*: a refusal that lands while the
connection is already excluded is the same event and is not counted.
- Nothing reset the streak on a healthy response, so sporadic refusals
on a busy install could still accumulate to a ban. chatHelpers'
onRequestSuccess now clears it (only a real success does - the recovery
tick's clearAccountError is an elapsed cooldown, not a success).
Clearing the cooldown by hand in the dashboard clears it too.
- The third rung of the ladder was unreachable (the third refusal
escalates): the ladder is now 5 -> 15 min, sourced from COOLDOWN_MS next
to the existing 5 s "request not allowed" rule, with a note on why that
rule is superseded for claude. The 60-min window becomes a 24 h
staleness bound - "consecutive" is defined by successes, not by time.
- Probe-origin refusals no longer touch the streak (#9817).
- The cooldown is written like every other connection-level cooldown:
ISO rateLimitedUntil + testStatus "unavailable" (+ lastErrorAt), so the
dashboard shows the countdown and the recovery tick restores "active".
- One refusal is re-seeded from the persisted row after a restart so a
crash loop cannot reset the count on every boot.
Docs: RESILIENCE_GUIDE terminal states + CODEBASE_DOCUMENTATION resilience
row mention the streak module. Tests cover the burst, the success reset,
the seed, and the ISO/unavailable shape end-to-end through the chat route.
* chore(sse): drop unrelated Prettier churn in auth.ts / providers route
* style(api): keep providers route Prettier-clean
* refactor(sse): share the "exclude connection for a cooldown" leaf between GEO_BLOCKED, GCP_PROJECT_REQUIRED and the new branch
The release tip moved chatCore.ts to its frozen 5984 lines, so the
REQUEST_REJECTED branch cannot add a single net line. The GEO_BLOCKED and
GCP_PROJECT_REQUIRED branches were the same eight statements with different
constants and log wording; both now call
open-sse/handlers/chatCore/connectionCooldown.ts::excludeConnectionForCooldown
(behaviour, probe guard and log lines preserved verbatim). chatCore.ts ends
9 lines below the base it branched from.
* chore(chatCore): tighten the cooldown comments to keep the file under its size ceiling
After merging release/v3.8.51, chatCore.ts sat at 6150 lines against a
frozen ceiling of 6146. Condense the explanatory comments this PR added
to the GEO_BLOCKED and GCP_PROJECT_REQUIRED branches; no code change.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: insoln <is@careerum.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(combo): answer 503 + Retry-After, not 404, when a weighted pool is only cooling down
The weighted strategy filters targets before dispatch (open circuit breaker,
provider cooldown, model lockout, availability probe) and drops them
silently. When that emptied the pool the host returned the 404
"Combo has no executable targets" with the "switch combo / reconnect the
missing providers" recovery hint — for a pool that was configured,
connected and merely cooling down. Claude Code renders a 404 from
/v1/messages as "this model may not exist".
- targetResolution.ts: the eligibility predicate now reports which gate
excluded a target and, for the resilience gates, the remaining time;
the exclusions of fully-excluded steps travel out of
resolveWeightedSelection. When the weighted pool ends empty and at least
one exclusion is a resilience timer, the pipeline returns an early 503
and logs the reasons at warn level.
- pinRecovery.ts: buildAllTargetsCoolingDownResponse() — 503
`all_targets_cooling_down`, Retry-After = earliest exclusion to lapse,
every excluded target in diagnostics.excluded, `wait` recovery hint with
retry_after_seconds; formatPreDispatchExclusions() for the log line.
- error.ts: `all_targets_cooling_down` joins the public error identifiers.
- A pool emptied only by the availability probe keeps the 404.
- docs: RESILIENCE_GUIDE debugging entry; changelog fragment.
* chore(changelog): name the fragment after PR #12956 and link issue #12954
* refactor(combo): keep weighted exhaustion below complexity ratchet
---------
Co-authored-by: insoln <is@careerum.com>
* fix(resilience): lock the exact model, not the quota family, on 5xx model-lockout failures
A 5xx model-lockout failure — a transport error (terminated, EHOSTUNREACH,
connect timeout), an upstream server error, or OmniRoute's own synthesized
502 from quality validation — is evidence about one model endpoint at that
moment, not about the account's quota family. recordModelLockoutFailure()
wrote it under the quota-family key regardless, so for codex (whose family
key is the whole `codex` scope, i.e. every gpt-5* model) one empty stream
on gpt-5.6-luna removed gpt-5.6-sol and gpt-5.6-terra from routing too,
for 2–30 min with exponential escalation, while the quota was untouched.
- exactModelLock.ts: resolveLockoutScope(status, explicit) — 429/403/402
(and 404, already narrowed by getModelLockKey) keep the family key; any
other status uses the exact provider/connection/model key. An explicit
`scope` option still wins.
- recordModelLockoutFailure() resolves the scope once for key + lock fn.
- decayModelFailureCount() now walks every key shape (family, not_found,
exact) so success-decay reaches exact-scope locks; null model stays a
no-op.
- getAllModelLockouts() parses the `exact:` marker out of the key so the
Model Cooldowns card lists the bare model and can clear it by that name.
- docs: RESILIENCE_GUIDE §3 key-scope-by-status; changelog fragment.
* chore(changelog): name the fragment after PR #12957 and link issue #12955
---------
Co-authored-by: insoln <is@careerum.com>
* fix(combos): accept isHidden in updateComboSchema
A combo's visibility is stored on the record and honoured by the builder
option list and the dashboard grid, but updateComboSchema never listed
isHidden. The PUT handler spreads the validated body, so zod stripped the
field: a visibility-only update was rejected as "No valid fields to
update", and a mixed update succeeded while dropping the visibility
change.
Closes#12836
* docs(changelog): fragment for #12898
* fix(resilience): honor declared effort vocabulary in reasoning rule gate
The reasoning-routing rule capabilityFor() hardcoded a gpt-5.6-(sol|terra|luna)
whitelist for forced max/ultra, rejecting every other thinking-capable model
even when the model's resolved capabilities declare the requested tier (synced
supportedThinkingEfforts or an operator Model Overrides reasoning_efforts
override). This 400'd direct calls with "Reasoning effort 'max' is not
supported by the configured target" for models like Merge Gateway
zai/glm-5.3-flash, which natively accepts low|high|max.
The gate now treats a declared vocabulary containing the requested tier as
authoritative, mirroring the dispatch-time sanitizer
(open-sse/executors/base/reasoningEffort.ts) which already forwards declared
tiers verbatim. Undeclared models keep the legacy gpt-5.6 regex verdicts and
the unknown passthrough.
* fix(resilience): gate forced max against the static registry the sanitizer clamps with
Adversarial review finding: the gate read supportedThinkingEfforts from
getResolvedModelCapabilities, which prefers the DB override over the registry.
For a registered model with a narrow registry vocabulary and a widening
operator override, the gate passed forced max but the dispatch-time sanitizer
(executors/base/reasoningEffort.ts) clamps against the STATIC registry and
would silently downgrade max to the registry ceiling — converting a loud 400
into a silent wrong-effort request.
Order of precedence in the gate now:
1. static registry vocabulary (authoritative — matches sanitizer clamping)
2. declared/overridden vocabulary for unregistered providers (#8057 path)
3. legacy gpt-5.6 regex, then unknown/unsupported verdicts
Also pins the test fixture to a synthetic model id so a future models.dev
sync row cannot flip the unknown-precondition assertion.
* fix(resilience): gate registry lookup mirrors the dispatch sanitizer exactly
Review findings on the forced max/ultra gate:
- resolve the registry through getProviderModels (id->alias namespace) and
match entry aliases, mirroring reasoningEffort.ts — a raw provider id or
alias-spelled model no longer skips the registry branch and diverges from
dispatch clamping
- treat an empty declared vocabulary as no declaration (falls through),
matching the sanitizer's declaredRanked.length>0 guard — before, a model
declaring [] was gated to unsupported while dispatch forwarded verbatim
- an operator-declared vocabulary that excludes the forced tier is terminal;
the legacy gpt-5.6 regex can no longer resurrect a tier the override
narrowed away
- rewrite the registry-outranks-override test: create the matching rule so
the decision is non-null, assert unconditionally, pin gpt-5.6 narrowing,
alias namespace parity, and use the deterministic xai/grok-4.6 fixture
* docs(changelog): clarify override scope for registry-declared models
* test: drop placeholder issue reference from test names
* chore(changelog): name fragment after PR #12686
OpenAI-compatible model discovery does not recognize per-vendor-route
reasoning vocabularies declared under vendors.<vendor>.capabilities.reasoning
in GET /v1/models (Merge Gateway's documented catalog shape), so synced
models carry no supportedThinkingEfforts/defaultThinkingEffort and operator
effort data resets on every model sync; models whose upstream accepts a
native max tier cannot be used with forced-max reasoning rules.
Parse the shape into the existing supportedThinkingEfforts pipeline,
intersected across vendor routes: the same canonical model declares
different vocabularies per route and unpinned requests self-narrow to a
route honoring the requested level, so a synced tier must be honored on
every route the model can land on. Routes without effort_values declare
no effort control and are excluded; disjoint vocabularies produce an
authoritative empty list (no fall-through to generic tier shapes).
detectDefaultThinkingEffort falls back to the intersection's highest tier
ranked by the canonical effort order — only when the vendors shape is the
record's winning vocabulary source, never escaping a flat or nested
declared list.
Detection is shape-gated, not provider-gated; Zod-validated (Hard Rule #7)
with malformed vendor and tier entries dropped individually (discarding a
whole route would widen the intersection, fail-open). Precedence: flat
field > reasoning.supported_efforts / metadata (#7694) > vendor-route
intersection > capabilities.effort_tiers (#9160) / supported_reasoning_levels
/ thinking.levels (#8347).
* security(deps): pin and verify tls-client native artifacts
* docs(changelog): link tls-client provenance PR
* security(runtime): harden TLS and public error boundaries
* security(runtime): resolve CodeQL error-boundary findings
* security(lmarena): close public stream error boundary
* fix(lmarena): normalize public error statuses
* chore(quality): rebaseline chatCore.ts for the surviving log-boundary hardening
open-sse/handlers/chatCore.ts 6219 -> 6287. This is the one part of #11742 that
survived the rebase: sanitizeErrorMessage on the plugin onError hook, on the
semaphore-timeout path and on failureMessage before it reaches console.log and
the call log, sanitizeUpstreamDetails on the malformed-response log, and
getSafeErrorMetadata + try/catch where hostile (Proxy) metadata could throw.
That is the LOG boundary, which is broader than Hard Rule #12 (responses). The
rest of the PR was dropped as already landed on the tip.
Measured on the clean tip (83fa4328), not on a branch. Each ceiling was
attributed to the PR that moved it before being raised — no blanket rebaseline:
- src/sse/handlers/chatHelpers.ts 1245 -> 1246 (#13551, combo scope on the
fail-closed proxy guard)
- open-sse/handlers/chatCore.ts 6203 -> 6219 (#12905 DSML/preamble, #13910
disguised-2xx classification, #12904 single post-translation system prompt)
- open-sse/utils/stream.ts 3123 -> 3140 (#12905, and #12906 empty_response 502
retry + reasoning-aware timeout)
- tests/integration/chat-pipeline.test.ts 1736 -> 1740 (#13419, the two exact
header assertions updated for charset=utf-8)
Other gates re-measured on the same tip and already green: typecheck:core,
check:open-sse-typecheck, check:docs-counts, and the #2331-adjacent
chatcore-translation-paths xhigh-effort case that was red before this wave.
* test(catalog): pin the 2026-09-02 free-tier re-audit facts for gemini, ollama-cloud, groq, nara and mistral
* feat(providers): add xKiro (5M tokens/day free plan, 39 pinned free models)
* fix(catalog): re-audit gemini, ollama-cloud, groq, nara and mistral against official pages
* docs(providers): xKiro in the provider reference, counts and free-tier headline (~1.66B)
* fix(catalog): restore the console-verified Mistral 1B pool and harden its regression test
* docs(free-tiers): move headline to the re-audited ~1.50B and refresh pool counts
* chore(free-tiers): retire stale Groq free-tier text and preset model; fix catalog header
* docs(providers): align the remaining visible provider/executor counts with the catalog
* docs(providers): align remaining free-tier count chips and metadata
* docs(free-tiers): state the evidence-comment rule honestly and retire the last "14.4K RPD" Groq texts
* docs(free-tiers): retire the stale Gemini onboarding quota text
* docs(free-tier): refresh catalog-entry counts to 442 after base sync
* docs(providers): re-sync provider and free-tier counts after merging release/v3.8.51
* docs(providers): re-sync residual counts after the base merge
* docs(free-tiers): restore README spacing lost in the merge and re-sync the guide counts
* docs(free-tiers): re-sync numbers after merging release/v3.8.51 (Cerebras reclassified upstream)
* fix(docs): keep the NaraRouter plans endpoint out of the API-path checker; rebaseline gateways.ts (+3)
* chore(quality): rebaseline gateways.ts file-size cap for the xKiro entry (+20)
---------
Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
* fix(providers): remove the chipotle/pepper provider (#13131)
amelia.chipotle.com (the reverse-engineered Amelia chat-widget backend
chipotle/pepper-1 talked to) now returns 404 on every route, including
root, from its Azure Application Gateway — confirmed live 2026-09-15.
This regressed from a WS handshake timeout (#4037, June 2026) to a
fully decommissioned host, so the upstream protocol cannot be fixed.
Owner decided to retire the provider entirely (Option B), following
the phind/kluster quiet-removal precedent: no REMOVED_PROVIDERS.md
entry (reserved for operator takedowns), just a one-line note under
FREE_TIERS.md "Removed / no free tier".
Removed every surface: executor, registry entry, executors/index.ts
and providers/index.ts wiring, noauth provider catalog entry,
ProviderIcon generic-fallback set, the autoCombo exclusion-list
comment, the chipotle_error code from the sanitizer allowlist,
PROVIDER_REFERENCE.md (regenerated), and every doc/test reference.
Regression test: tests/unit/issue-13131-chipotle-provider-removed.test.ts
asserts the provider is fully gone from the executor registry, the
provider REGISTRY and the noauth catalog, and that the executor module
no longer resolves — not a live-network repro (flaky/third-party).
Several existing tests used "chipotle" only as a generic noAuth-provider
example (proxy scoping, error classification, onboarding, fallback
text) with no chipotle-specific behavior under test; those were
re-pointed at another still-existing noAuth provider
(cloudflare-playground / duckduckgo-web) rather than weakened.
* test(providers): document the agnes-cn/chipotle count coincidence (#13131)
provider-node-reserved-prefix.test.ts's REGISTRY id+alias walk was
already red on the base tip (414 vs. expected 412) from agnes-cn
(#13399, +id/+alias). Removing chipotle's REGISTRY id/alias in this
PR nets it back to 412, making the test pass again without a numeric
edit — record why in a comment so it doesn't read as an untracked
coincidence later.
* fix(chat): reject null/non-object entries in messages[] (#12643)
A messages array containing null (or any non-object entry, e.g. [null] or
[42]) passed every existing entry guard in chat.ts (#5110/#6402/#6407/#6412)
and reached downstream translators/session helpers that read `.role` /
`.content` directly off each entry (openai-to-claude.ts, sessionManager.ts,
contextManager.ts's fixToolPairs), crashing with a raw TypeError and
surfacing as an HTTP 500 instead of a clean 400. The route's Zod schema is
intentionally wide (z.array(z.unknown())), so this shape check belongs in
the handler's guard chain. Adds one more entry-shape guard clause to the
same chokepoint, rejecting the request with a clear 400 before any routing
or upstream call.
Regression test: tests/unit/chat-messages-entry-objects-12643.test.ts
PR #12644 (@soroush5) proposed this exact fix but was closed without
merging on 2026-09-12; this re-implements it fresh against the current tip
using the same guard shape and error message.
Originally-proposed-by: @soroush5 in #12644
Co-authored-by: soroush5 <mrsoroushahmadi@gmail.com>
* chore(quality): refix the chat.ts ceiling for the merged tree
This branch rebaselined src/sse/handlers/chat.ts against an older tip. After
merging the current release tip the combined file is 2520 lines, so the 2500
ceiling no longer covers it.
The tip alone is already at 2509 — above the 2500 this PR had frozen — so most
of the gap is inherited, not introduced here. This PR's own contribution is the
+10 of the messages-entry guard itself. Ceiling refixed at the value the gate
reports for the merged tree.
---------
Co-authored-by: soroush5 <mrsoroushahmadi@gmail.com>
On 2026-09-16 eight feature PRs added 61 keys to src/i18n/messages/en.json and
stamped `__MISSING__:<en>` into all 65 locales instead of translating.
check-new-key-coverage accepted the marker as "the key reached the locale", so
nothing blocked the PRs, and the blocking real-translation ratio gate then failed
on the release tip for everybody (pt-BR 3.2 % > 2.5 % + 0.5).
- scripts/i18n/check-new-key-coverage.mjs: a leaf whose value starts with
`__MISSING__:` is judged exactly like an absent leaf; the FAIL message names
the marker as the cause and prints the per-locale sync-ui-keys command and the
parallel runner. Header/JSDoc updated.
- tests/unit/i18n-new-key-coverage.test.ts: "a new key that only carries a
__MISSING__ marker is flagged" (was the inverse case, which encoded the old
contract); the other six cases unchanged and green.
- scripts/i18n/translate-new-keys.sh (+ `npm run i18n:translate-new-keys`):
committed, detached-safe runner — flock queue, N workers (default 5), 3
attempts per locale of `sync-ui-keys.mjs --translate-markers --batch-size=40`,
per-locale logs/.exit + batch.log/batch.status/batch.rc/batch.pid under
_artifacts/i18n-new-keys/, non-zero exit while any locale still carries a
marker, refuses to start (exit 2, names the five OMNIROUTE_TRANSLATION_* vars)
when the backend env is absent. Reads only the OMNIROUTE_TRANSLATION_* lines
of the repo .env; kills nothing, matches nothing by name.
- docs: QUALITY_GATES.md (gate table + check-new-key-coverage section) and
I18N.md (gate table + "Translating the keys a branch adds" subsection).
The implementation/port/merge skills reference the new shared snippet
`.agents/skills/_shared/i18n-translate-new-keys.md` (skills repo, separate).
Root cause: GeminiWebExecutor.execute() opened the identical fixed
https://gemini.google.com/app URL and ran the identical Playwright
interaction sequence for every advertised gweb/<model> id. `model` was
read only AFTER the response was captured, purely to stamp the
OpenAI-shaped response — never to influence what was actually
clicked/typed, so two different advertised models produced
byte-identical automation and the response `model` field was a
caller-supplied label, not an observed fact.
Fix (owner decision, Option B): a new model -> Gemini UI mode map
(open-sse/executors/gemini-web/modeSelection.ts) drives an in-browser
selection step before anything is typed — try the mode control, read
back the active-mode indicator, and only proceed on a confirmed match.
An unconfirmed model, or a requested Extended Thinking control that
cannot be confirmed (#13381 follow-up comment), fails closed with 400
unsupported_control_for_provider instead of silently running the
account default under the requested label. The selectors involved are
UNVALIDATED (no live Gemini account from this checkout) — see the PR's
"Selector set is UNVALIDATED" section and the required live smoke.
Regression test: tests/unit/issue-13381-gemini-web-model-selection.test.ts
#13661 bumped the exact `bun` devDependency (and `@types/bun`) from 1.4.0
to 1.4.2, and #12977 moved the `oven/bun` image to 1.4.2-slim. AGENTS.md
still documented 1.4.0, so the guide disagreed with package.json for
every agent reading it as authority.
Version string only. The surrounding policy is deliberately unchanged:
Bun stays confined to the allow-listed gate/generator scripts plus the
`test:bun:db` smoke suite, and Node remains the only supported runtime.
Operator approved editing this protected surface for this change.
* fix(thinking): recognize adaptive thinking + parse/scrub DSML tool-call markers
Two defects combined to break DeepSeek-V4-Flash turns and raise 502
empty_response on Claude Code autocompact.
Defect 1 — DSML tool-call markers leaked as visible content:
DeepSeek-V4-Flash occasionally emits tool calls in a non-standard DSML
text format using full-width pipes instead of the OpenAI tool_calls JSON.
Two shapes appear in production call logs:
- complete block: <｜DSML｜:Read><path>...</path></｜DSML｜:Read>
- stray closers (truncated call): </｜DSML｜parameter></｜DSML｜invoke>
</｜DSML｜tool_calls>, sometimes trailing a system-prompt echo
The openai-compatible path never parsed these, so the markers leaked to
the client as visible content and the turn ended incomplete.
Fix: add open-sse/utils/dsmlToolCalls.ts — parseDsmlToolCalls() converts
complete DSML blocks into OpenAI tool_calls and strips stray closing
markers from content (streaming-safe via a holdback for partial openers).
Wire it into the response translator before extractXmlInvokeBlocks so
DSML and XML invoke tool calls share the same pending queue.
Defect 2 — adaptive thinking silently suppressed:
A prior inline === 'enabled' check on body.thinking.type silently
suppressed adaptive (the intent Claude Code actually sends), so
reasoning was dropped. The model then emitted DSML tool-call markers
as plain text, producing an incomplete stop finish. Fix: use
hasActiveClaudeThinking() (which recognizes enabled AND adaptive) to
set requestedThinking, thread it through stream.ts and translator
state, and gate thinking block emission on state.requestedThinking
so upstream reasoning_content only relays when the client opted in.
Tests: 29/29 (6 dsml-tool-calls, 5 thinking-active-claude-adapter,
3 translator-resp-dsml-integration, 15 translator-resp-openai-to-claude
incl. requestedThinking suppression regression). typecheck:core clean.
* fix(sse): strip echoed system-prompt preamble + preserve large analysis/summary blocks
DeepSeek-V4 and similar models echo the OMNIROUTE_SYSTEM_INSTRUCTION_APPEND
directive (appended to the system tail by claude-to-openai.ts) and whole chunks
of the system prompt (<analysis>/<system-reminder>/<summary> blocks, prose
reproductions of the superpowers skill section) verbatim at the START of their
reply — the 'system message leak' persisting after the request-side fix.
Add two streaming-safe preamble strippers in directivePreambleStripper.ts:
- createDirectivePreambleStripper(directive): drops a leading reproduction of
the exact configured directive across arbitrary SSE chunk boundaries.
- createSystemPreambleStripper(): removes <analysis>/<system-reminder>/
<summary> echo blocks and known prose heads (Phase B) from the very start
of a stream, only while the stream is still a preamble.
Wire both into openai-to-claude.ts content-delta path: chain the exact-directive
stripper then the system-echo stripper before DSML/XML-invoke parsing, so a
leading system echo is dropped before it reaches the client.
Preserve large blocks (>= SYSTEM_ECHO_THRESHOLD=1000 chars) and blocks with no
trailing content — these are the model's real response (e.g. a Claude Code
autocompact summary), not a short system-echo. Stops the autocompact
empty-response regression where a whole-summary <analysis> block was stripped
to empty (3a8515).
Regression: origin's markdown-boundary feature (bufferedPrefix /
splitMarkdownBoundary, commit 1b39873ea) is preserved — preamble strip runs
before the markdown buffer rehydration, and the scrubbed content flows into
the existing DSML/XML-invoke/markdown pipeline unchanged.
TDD: tests/unit/directive-preamble-strip.test.ts (7 cases),
system-preamble-strip.test.ts (12 cases incl. 3a8515 regression),
system-preamble-wiring.test.ts (3 integration cases); group F regression
24/24 green; typecheck:core 0 errors.
* fix(sse): gate thinking block on requestedThinking + synthesize text block for reasoning-only responses
Reasoning-content (thinking) blocks were emitted unconditionally to
Claude-format clients, leaking reasoning to thinking-opt-out clients
(Claude Code sends thinking:{type:"disabled"}) — the operator reported
'reasoning is exposed'. On reasoning-only upstream responses (GLM-5.2
autocompact pattern), the unconditional thinking block also caused either
a 502 'no content block' at flush, or — after a text-block fallback — an
autocompact 'empty response' rejection that looped the session forever.
Streaming translator (openai-to-claude.ts):
- Compute hasReasoning outside the emission gate; accumulate into
state._reasoningAccum always (so fix B can fire).
- Gate only the thinking-block EMISSION on requestedThinking === true.
- FIX B at finish: when no text block was started and requestedThinking
!== true, synthesize a text content block from _reasoningAccum so
autocompact can extract the summary (no 502, compact applies).
- Skip fix B when requestedThinking === true to avoid double-exposure
(thinking block + text block both carrying reasoning).
Non-streaming translator (responseTranslator.ts):
- Thread requestedThinking through translateNonStreamingResponse into
convertOpenAINonStreamingToClaude.
- suppressThinking = requestedThinking === false: drop the thinking block
when content is present (no leak); relay reasoning as a text block when
the response is reasoning-only (no 502). requestedThinking === undefined
keeps the legacy 'always a thinking block' relay.
chatCore.ts: pass hasActiveClaudeThinking(body) to the non-stream
translate call (inline, since the shared const is in the stream branch's
temporal dead zone here).
Tests: 25/25 (5 gate-restore, 1 gate-502-repro, 4 nonstream-leak,
15 resp-openai-to-claude incl. requestedThinking suppression regression).
Group E (22) + F (14) regression-free. typecheck:core clean.
* fix(sse): restore requestToolIdentityMap in the Codex CLI responses-translation path
The needsResponsesTranslation branch (openai-responses -> openai, used
when the client also speaks Responses) silently dropped the
requestToolIdentityMap argument to createSSETransformStreamWithLogger
when the requestedThinking parameter was added, reverting the #7936
tool-identity round-trip fix for that branch. The sibling
needsTranslation branch was updated correctly; restore the same
argument here.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* docs(changelog): add the 3 fragments documented in the PR body
The PR body already writes out the changelog.d/ entries for the DSML
parser (Group F), the directive-preamble stripper (Group E), and the
reasoning-gate thinking-leak fix (Group G), but none of the files
existed in the diff. Add them so the release aggregator picks them up.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(sse): realign GLM's positional call after the new requestedThinking parameter
createSSETransformStreamWithLogger gained a new requestedThinking
parameter inserted before customToolNames. glm.ts's translateSseResponse
still called it with the pre-existing positional argument list, so the
new parameter silently absorbed the old customToolNames slot, and the
GLM_STREAM_BUFFER_BYTES tuning value (#12925) landed on
requestToolIdentityMap instead of streamBufferBytes — a TS2345 (number
is not assignable to Map<...> | null) caught by
check:open-sse-typecheck, and a real loss of GLM's 64KB stream buffer
budget. Insert an explicit `undefined` for requestedThinking to restore
the original alignment.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(sse): make the system-preamble stripper opt-in and flush it at stream end
`createSystemPreambleStripper()` was wired DEFAULT-ON and unconditional in the
openai→claude streaming translator, unlike the exact-directive stripper right
above it, which only runs when the operator configured
OMNIROUTE_SYSTEM_INSTRUCTION_APPEND. Cause: the system-echo stripper recognises
its openers by English-prose heuristics ("# Skill usage", "# Verification
Process", <analysis>/<summary>/<system-reminder>), so leaving it always-on made
it mutate the response payload of EVERY openai→claude stream. A legitimate reply
opening with "# Skill usage: how to write one\n\nHere is the guide." lost that
whole section. It is now gated on OMNIROUTE_STRIP_SYSTEM_PREAMBLE=1, mirroring
the directive stripper's opt-in.
Second cause, same feature: neither stripper was ever flushed. Both buffer while
a construct is still undecided — a directive prefix that never completes, an
<analysis> block that never closes — and nothing released that buffer at the end
of the stream. A reply consisting of an unterminated echo block therefore reached
the client as an EMPTY message: the answer was held in the buffer and discarded
with the stripper. Both strippers now expose flush(), the finish handler calls it
for both, and the released text is emitted as a text block. A construct that WAS
finally classified as an echo is not resurrected (the drop is final).
Tests: tests/unit/system-preamble-gate-and-flush.test.ts pins the default-off
contract, the opted-in behaviour, the flush for both strippers (unit + wiring),
and the no-resurrection guard. system-preamble-wiring.test.ts now opts in
explicitly, since it exercises the stripping path.
* fix(sse): thread the client's thinking intent into the non-streaming path
The streaming and non-streaming translators disagreed on the default meaning of
`requestedThinking`, so the SAME request produced different shapes depending on
`stream`. Cause: chatCore computes the client's intent
(hasActiveClaudeThinking) and threads it into the SSE translator, which relays
reasoning as a thinking block only when it is explicitly `true` — but NO caller
ever passed it to translateNonStreamingResponse(). The non-streaming
OpenAI→Claude conversion therefore only ever saw `undefined`, its legacy
"always relay a thinking block" default, and leaked reasoning to a client that
had opted out with `thinking: {"type":"disabled"}`. The streaming plumbing also
coerced an omitted value into an explicit `false`, hiding the divergence behind
two different spellings of "no intent".
Fix (least destructive of the options): do NOT flip either gate — both encode a
deliberate, regression-tested contract — but give the non-streaming path the
same input the streaming path already has. runNonStreamingProviderLeg owns the
client body (`sourceBody`), so it computes the intent with the very same helper
and passes it down through translateNonStreamingClientResponse. `undefined`
keeps its documented back-compat relay for callers that cannot express intent
(issue-7856 / issue-6623), and stream.ts no longer defaults the parameter to
`false`, so "absent" now means the same thing in both signatures.
No content is lost by the suppression: a reasoning-ONLY response is still
relayed as an ordinary text block (no empty response, no 502) — exactly what the
streaming finish handler does.
Tests: tests/unit/nonstream-requested-thinking-parity.test.ts drives the real
provider leg with thinking disabled / enabled / adaptive.
---------
Co-authored-by: Jihyun Son <jihyun.son@sk.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(claude-web): add charset=utf-8 to Content-Type headers to fix Arabic/Persian UTF-8 mojibake
Fixes#13416
The Claude Web endpoint and the outer SSE streaming pipeline were returning
Content-Type headers without an explicit charset parameter:
- stream.ts responseHeaders() returned 'application/json' and
'text/event-stream' without charset
- responseHeaders.ts buildStreamingResponseHeaders() returned
'text/event-stream' without charset
While RFC 8259 defaults JSON to UTF-8 and the SSE spec defaults
text/event-stream to UTF-8, some HTTP clients (notably VS Code Chat on
Windows) fall back to ISO-8859-1/Latin-1 when no charset is declared,
causing multi-byte UTF-8 characters to appear as mojibake.
For example, the Persian word for hello (سلام, UTF-8 bytes D8 B3 D9 84 D8 A7
D9 85) was decoded as Latin-1, producing the garbled output 'سلام'.
Fix:
- claude-web/stream.ts: append '; charset=utf-8' to the Content-Type
header in the responseHeaders() helper, with a guard to avoid double
appending if the caller already includes a charset
- chatCore/responseHeaders.ts: hardcode 'text/event-stream; charset=utf-8'
in buildStreamingResponseHeaders()
Tests:
- 11 new regression tests in claude-web-utf8-mojibake-13416.test.ts
covering Persian, Arabic, mixed-script, emoji, and chunk-boundary-split
scenarios across both streaming and buffered response paths
- All 7 existing claude-web-stream tests pass
- All 13 response header tests pass
- All 8 adaptive-admission-lifecycle tests pass
* test(sse): confirm streaming charset header and add byte-level UTF-8 repro (#13416)
Independently verified the mojibake root cause before trusting the charset
fix: OmniRoute's claude-web decoder already reconstructs a Persian/Arabic
multi-byte UTF-8 sequence split across a chunk boundary correctly because
it decodes with TextDecoder({ stream: true }); a naive per-chunk decode
(without stream state) is what actually produces the U+FFFD garbling.
Updated the 2 exact Content-Type assertions in chat-pipeline.test.ts to
match the new "text/event-stream; charset=utf-8" header, which is already
the convention used by the other streaming executors (uc.ts, maxai.ts,
codex-app-server.ts, etc).
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: Koosha Pari <koosha@phenotype.ai>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(db): union customModels with syncedAvailableModels in dispatch path
* fix(compression): skip RTK dedup and truncation for non-shell tool results
RTK's line deduplication and truncation were applied to all tool results
including non-shell tools (read, grep, glob, edit, write). This collapsed
structurally meaningful repeated lines in file content (e.g. JSON closing
braces, repeated key names), silently corrupting what the model received.
Now skipFilters (set for non-shell tools) and isDocumentLikeRead both
gate dedup and truncation, so file content survives byte-identical.
Fixes#13388
* fix(compression): restrict RTK truncation skip to document-like reads
The non-shell truncation-skip (options.skipFilters) disabled the generic
line/char cap for every non-shell tool result, including grep/glob/search
output that #4559 deliberately did NOT exempt. Only isDocumentLikeRead now
gates the generic truncation cap; the broader skip stays for dedup, which
is the operation that actually corrupts structured JSON content.
Also drops docs/omniroute-pr-body.md, an out-of-scope file carried by an
unrelated commit on this branch, and fixes the regression test's broken
relative import (tests/unit/compression -> open-sse is 3 levels up, not
2 — this is why the test file could not even load before this commit),
adjusts its truncation fixture to a genuinely document-like (non-JSON)
read so it actually exercises the isDocumentLikeRead exemption, and adds
a negative case asserting large non-shell grep output still gets
truncated by the generic cap.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* test(compression): make the RTK preservation test an actual regression guard
The four cases all passed against the tip WITHOUT this branch's fix, so they
guarded nothing — the next RTK refactor could reopen#13388 in silence.
Two causes, both fixed here:
- The fixture's repeated lines were not consecutive, and deduplicateRepeatedLines
only collapses consecutive runs, so dedup never ran on it. Replaced with a
matrix of identical rows, where collapsing them CORRUPTS the data rather than
just reformatting it — which is the damage the fix prevents.
- Both central assertions sat inside `if (result.stats)`. With the old fixture
the engine reported no stats, so the assertion bodies were skipped entirely
and the test passed by doing nothing. They now run unconditionally.
Verified in both directions on the current tip:
with this branch's fix → tests 4 | pass 4 | fail 0
against the tip's engine → tests 4 | pass 3 | fail 1
✖ RTK should NOT dedup file content from a
non-shell 'read' tool
---------
Co-authored-by: Forge <forge@kooshapari.local>
Co-authored-by: KooshaPari <kooshapari@users.noreply.github.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(compression): remove isStrictlySerializable gate that rejects valid bodies (#13154)
Fixes#13154
isCompressionWorkerEligible used isStrictlySerializable to pre-validate
bodies before posting them to a worker thread. The gate is stricter than
structuredClone (which postMessage uses natively), rejecting:
- undefined values (common in optional config fields)
- Date, Map, Set, Uint8Array, RegExp (all structuredClone-compatible)
- Shared (non-cyclic) sub-objects (misread as cycles)
This caused compression to fall back to inline execution on the main
event loop, blocking every concurrent request for the duration of
compression passes — the exact failure mode of #10300.
The serializability walk is a slower, buggier duplicate of the check
postMessage already performs. Removing it:
- Eliminates a recursive walk of the entire body on the main thread
- Fixes false rejections that prevent worker offload
- Allows the catch block at the call site to properly fall through
to inline compression instead of silently shipping uncompressed
* fix(compression): track only the recursion path, not the whole tree, in isStrictlySerializable (#13154)
Restores the cycle-detection gate instead of removing it: the original
bug was a single `seen` set shared across the entire recursion tree,
never backtracked, so two sibling branches referencing the SAME
non-cyclic sub-object were misread as a cycle. Adding to `seen` before
descending and removing it after (try/finally) fixes the false positive
while a genuine cycle is still rejected before it ever reaches
postMessage/the worker.
Also reverts the worker-failure catch in runCompressionAsync back to
returning the body uncompressed: a worker timeout means the compression
was already too heavy for the worker's own budget, so falling through
to run that same heavy compression synchronously on the main event loop
defeats the point of offloading it to a worker in the first place.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: Koosha Pari <koosha@phenotype.ai>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(api): stream /api/logs/export with row cap to prevent V8 heap OOM (#13123)
Fixes#13123
GET /api/logs/export buffered every matching row into a single
JSON.stringify call with pretty-printing (null,2), roughly doubling the
string size. On tables with tens of thousands of rows this crashed the
Node process with a V8 heap OOM, taking the gateway down for minutes.
Changes:
- Stream the response via ReadableStream, serializing one row at a time
so peak memory stays bounded regardless of table size.
- Add a configurable row cap (limit query param, default 10000, max
50000) so callers cannot accidentally request unbounded exports.
- Remove pretty-printing (callers can pretty-print client-side).
- Include cap metadata (capped, limit, totalAvailable) when the cap
fires so callers know they received a truncated result.
- Preserve backward-compatible response envelope: { count, hours, type,
logs, ... }.
* fix(api): push the /api/logs/export row cap down into the DB layer (#13123)
The route-layer streaming + cap from the previous pass still called
exportCallLogsSince()/exportProxyLogsSince(), which hydrated and
buffered EVERY matching row (including rows beyond the limit) before
the cap was ever applied — peak V8 heap was essentially unchanged.
Adds countCallLogsSince()/countProxyLogsSince() (cheap COUNT(*), no row
hydration, used for totalAvailable) and iterateCallLogsSince()/
iterateProxyLogsSince() that bound the query with SQL LIMIT and
yield/hydrate one row at a time: a generator over a LIMIT-bounded id
list for call_logs, and fixed-size LIMIT/OFFSET pages for proxy_logs
(the shared SqliteAdapter only exposes run/get/all, not a `.iterate()`
cursor, so LIMIT/OFFSET pagination is the available cursor-equivalent
without widening that interface across all 4 driver adapters). The
route now streams from these instead, so the full matching row set is
never buffered.
Also moves capped/limit/totalAvailable into the response header instead
of only the trailer, so a client consuming the stream incrementally
learns about truncation before processing every row.
Rewrote the test to call the real route.GET handler against a seeded
test database instead of a local reimplementation of the stream
builder, so a regression in the route or its DB-layer delegates is
actually caught.
Documents the pre-existing (now more clearly load-bearing) breaking
change in a changelog fragment: `limit` defaults to 10,000 rows, so
exports that previously returned everything are silently truncated.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: Koosha Pari <koosha@phenotype.ai>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Second `npm run release:reconcile` pass on `release/v3.8.50..release/v3.8.51`
(091589089c..c0f92ec98a, 916 non-merge commits, 877 merged PRs):
- fold the 173 changelog.d fragments accumulated since #12971 under
`## [3.8.51]` and delete them
- generate bullets for the 58 cycle commits that had no fragment
(4 features / 42 fixes / 12 maintenance), each with the merged PR link and
`— thanks @author`
- link 137 fragment bullets to the PR of the commit that added them and
credit the author; two prefix/origin mismatches reviewed (#12945→#13392,
#13001→#13379, both maintainer rebaselines of other people's PRs)
- refresh "Release by the numbers" + Top-25 and regenerate the
`### 🙌 Contributors` hall (112 external contributors + maintainer; every
non-bot author of the 877 merged PRs present)
- closed-PR credit audit for the window: nothing to add (#13215→#13361 and
#13059→#13690 are still open, #12998 was independently fixed earlier by
#12853); no human co-author trailers, no commits without a PR
- resync the 58 i18n CHANGELOG mirrors
Gates: check:changelog-integrity OK, check:docs-sync PASS.
Root cause: when a user's Augment/Auggie quota is exhausted, the local
`auggie` CLI prints its "You have run out of usage for ..." warning to
stdout and exits with code 0. AuggieExecutor treated any clean exit as
a successful completion and wrapped that text as a normal 200 assistant
reply, so combo/fallback routing never saw a failure and kept sending
requests to the same exhausted connection.
Fix: detect the CLI's known quota-exhausted phrasing before wrapping
stdout as a completion. Non-streaming returns a 429 in-band error body
(mirroring blackbox-web.ts's precedent for HTTP-200 in-band errors);
streaming buffers the first ~2KB of stdout, and on a match emits the
existing {error:...} SSE envelope (reusing the #7880 combo quality-gate
detection) instead of forwarding the text as a delta.
Regression test: tests/unit/issue-12949-auggie-quota-exhausted-200.test.ts
Root cause: resolveRequestToolIdentity (the #7936/#9780 seam) only ever
resolved a streamed/non-streamed Responses function_call item's
{namespace, name} from the CURRENT request's own requestToolIdentityMap,
built solely from that request's own tools:[{type:"namespace",...}]
declarations. OmniRoute is a stateless-upstream-by-default proxy for the
Responses API, so nothing persisted namespace identity across separate
top-level HTTP requests in the same Codex/MCP session — a follow-up turn
that relies on previous_response_id/session continuity instead of
re-declaring its namespace tools silently lost the namespace field.
Fix: a deterministic wire-name fallback that splits a flattened
mcp__-namespaced wire name on its last "__" separator (mirroring
flattenNamespaceToolName's own construction), scoped to the mcp__
container-name convention so it cannot misfire on an unrelated flat tool
name that happens to contain "__". Applied symmetrically to the
non-streaming path.
Regression test: tests/unit/issue-12996-responses-streaming-namespace-followup.test.ts
tests/unit/9147-catalog-eventloop-yield.test.ts exists to prove the /v1/models
builder yields to the event loop while assembling a catalog-scale dataset. It
asserts res.status === 200 first, and only then the two checks that carry the
invariant: the max event-loop gap and the traversal to the last seeded model.
The case did not set CATALOG_BUILD_TIMEOUT_MS, so it inherited production's 8s
cold-path budget. When the seeded build overruns that on a loaded runner,
getUnifiedModelsResponse answers 503 catalog_build_timeout and the status check
fails BEFORE either real assertion runs — the guard goes silently dead exactly
when the machine is under the load that would make a pin most visible. CI hit
it at 8350ms, right at the bound.
Measured on the release tip, pristine file, 3 runs at load ~21: 2 pass with max
gaps of 247ms and 180ms, 1 fails with 503 !== 200 — roughly a 1-in-3 flake, and
the flake has nothing to do with yielding.
Pinning a 120s budget lets the invariant be evaluated. The 800ms gap bound is
untouched, and with the budget pinned the builder measures 261ms (idle) to
790ms (loaded) against it — still failing a true pin, which is seconds.
Build-latency budgeting is a separate concern from this case.
Refs #12732
Root cause: ServerSupervisor.handleExit()'s give-up branch only printed the
crash summary/log via console.error(). In --tray/--tray-worker mode this
process is launched detached with stdio:"ignore" on Windows and Linux
(bin/cli/tray/detachedTray.mjs buildTrayLaunch()), so that output is
discarded by the OS and nothing ever explained why the tray + gateway
disappeared together.
Fix (Part A only, see plan-file for Part B — a restart-on-failure policy,
out of scope here): best-effort append the crash summary + buffered log to
<DATA_DIR>/server/crash.log, wrapped in try/catch so the write can never
block shutdown. Surface the same file from `omniroute doctor`.
Regression test: tests/unit/issue-13538-tray-crash-diagnostics-lost.test.ts
Both src/lib/machineToken.ts::getActiveSalt() and its mirror in
bin/cli/utils/cliToken.mjs derived the CLI/management bearer token as
HMAC-SHA256(raw machine-id, salt) with a checked-in literal default salt
("omniroute-cli-auth-v1"). Since /etc/machine-id is commonly world-readable,
any local user who never set OMNIROUTE_CLI_SALT could derive the same
bearer token as the server.
getActiveSalt() now generates a random 64-char-hex salt on first use and
persists it under <DATA_DIR>/cli-token-salt.json (falling back to the
literal only when neither the env override nor a persisted/writable salt
can be established). Both implementations use the same resolution order
and the same wx-flag create-race handling so the CLI and server keep
deriving the same token. OMNIROUTE_CLI_SALT stays the explicit operator
override, unchanged.
Regression test: tests/unit/machine-token-random-salt-13679.test.ts
Pollinations and Perplexity-web can answer a genuine failure (expired
session, exhausted free-tier credits) with HTTP 200 and a structurally
normal completion whose assistant text is just the provider's own
error prose. classifyProviderError() only inspects the body for
400/401/402/403/429, and detectMalformedNonStream() only checked
structural emptiness, so the error text reached the client as a real
answer and combo/auto-fallback never triggered.
Adds classifyFakeSuccessBody() in errorClassifier.ts — allowlisted to
pollinations/perplexity-web, reusing the existing
CREDITS_EXHAUSTED_SIGNALS/ACCOUNT_DEACTIVATED_SIGNALS phrase lists,
gated on short content with a dominant signal match — and wires it
into detectMalformedNonStream() so the existing malformed-200 /
combo-failover path picks it up with no other handler changes.
Regression test: tests/unit/diagnostics-fake-success-13461.test.ts
PR E of the #13679 insecure-defaults umbrella (items #6, #7; item #8 analyzed
as by-design, no change). The published Docker image and fly.toml shipped
without REQUIRE_API_KEY set, so a bare `docker run` (README/QUICK-START
one-liners, no --env-file) or a `fly deploy` combined "keyless" with
"world-reachable" for the anonymous /v1 LLM proxy. docker-compose.yml already
mitigates this via loopback-only binding (#12568) and correctly keeps
following the operator's own .env, so it is untouched. The npm/CLI
local-first REQUIRE_API_KEY=false default in featureFlagDefinitions.ts is
also untouched per the owner's decision.
/api/free-tier/summary ships an unconditional Access-Control-Allow-Origin: "*"
and always included the operator's own local usedThisMonth/remaining usage
regardless of auth — a low-severity info leak to any reachable origin. Both
fields are now withheld from unauthenticated callers while the intentionally
public catalog data stays served to everyone.
The gemini-SSE (openai-to-gemini-sse.ts) sub-finding needed no code change:
/v1beta/models/*:streamGenerateContent is already classified CLIENT_API and
fronted by clientApiPolicy through src/proxy.ts before the translator ever
runs, and its CORS-header echo was already hardened fail-closed by #12573.
REQUIRE_API_KEY=true (this PR's container/Fly default) closes the dependency
that finding cited. Added a locking regression test confirming this chain.
Regression tests:
- tests/unit/issue-13679-container-posture-require-api-key.test.ts
- tests/unit/issue-13679-free-tier-summary-usage-leak.test.ts
- tests/unit/issue-13679-gemini-sse-requires-api-key.test.ts (confirmation)
Refs #13679
* fix(sse): bound active streams without terminal events
* fix(sse): derive the active-stream ceiling from the largest registered model budget
The watchdog is a hard lifetime cap that never resets on bytes, so a flat
15-minute default killed models the registry already allows to run for 20
minutes (the Codex entries declare timeoutMs: 1_200_000). The default is now
that maximum plus a one-minute margin, and a new test re-derives the maximum
from the registry so a future larger budget fails the gate instead of silently
re-opening the bug.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(resilience): allow maxWaitMs=0 as disable sentinel for execution expiration
maxWaitMs normalization clamped the value to min:1, silently rewriting
an operator's 0 ("disable the limiter-managed execution deadline") into
1 — a 1ms expiration that killed every long-running job instantly. This
broke long-running reasoning models (GLM-5.2 with reasoning.effort=max
spends minutes before the first token, exceeding any practical
maxWaitMs; the TTB safety net is FETCH_TIMEOUT_MS, default 600s).
Fix: lower the floor to min:0 so 0 is preserved as the disable sentinel.
Issue #4165 follow-up.
Tests: 7/7 (resilience-normalize-maxwaitms-disable 5 + rate-limit-
maxwaitms-disable-execution 2). typecheck:core clean.
* fix(resilience): relax requestQueueSettingsSchema.maxWaitMs to allow 0
normalizeRequestQueueSettings already treats maxWaitMs=0 as an explicit
disable sentinel (queue-wait budget off), but the settings API schema
still rejected 0 with min(1), so an operator could never actually reach
the fix through PATCH /api/resilience. executionMaxWaitMs is untouched
(stays min(1) — separate field, separate decision, see #12902 item 4).
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* test(resilience): prove maxWaitMs=0 vs #12715's queue-wait gate behavior
Answers the open technical question from #12902's review: does a
GLOBAL maxWaitMs=0 reintroduce the unbounded-queue regression #12715
fixed (a request hanging ~6min until the client aborts)?
Evidence, exercising the real gate chatCore.ts actually calls
(accountSemaphore.acquireMany({ timeoutMs: requestQueue.maxWaitMs }),
not the Bottleneck reservoir the PR's own tests cover) under real
contention (maxConcurrency=1, two concurrent acquires):
- No: it does not hang. setTimeout(reject, 0) fires on the next
tick, so a second contending request is rejected with
SEMAPHORE_TIMEOUT in low milliseconds, never minutes.
- But it is also not a genuine 'no cap' — an operator setting 0
expecting 'wait as long as it takes' instead gets near-zero
tolerance for even momentary contention on any configured
concurrency gate (global/provider/account). This is a real
asymmetry vs. the Bottleneck reservoir path (where 0 truly means
unbounded) left for the maintainer to decide how to resolve —
not something this pass can decide unilaterally.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: Jihyun Son <jihyun.son@sk.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(proxy): add combo scope to fail-closed proxy guard (fixes#13469)
The hasBlockingProxyAssignment guard only checked account, provider, and
global scopes. Combo-scoped proxy assignments were not checked, so a fully
dead combo pool fell through to direct egress — leaking the host IP.
- Add combo scope to the SQL guard query
- Add optional comboName parameter to hasBlockingProxyAssignment
- A dead combo pool now blocks egress like the other three scopes
* fix(proxy): thread comboName through safeResolveProxy to the combo-scope guard (#13469)
hasBlockingProxyAssignment() gained a comboName parameter and a combo-scope
SQL clause, but its only caller, safeResolveProxy() in chatHelpers.ts, never
passed it — the clause always bound NULL and never matched a real combo
scope_id, so a fully dead combo-scoped proxy pool still fell through to
direct egress. Thread comboName from handleSingleModelChat (where it is
already in scope) through safeResolveProxy into the guard, and add tests
covering both the guard predicate and the end-to-end wiring.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: Koosha Pari <koosha@phenotype.ai>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* feat(sse): retry transient 5xx backend errors with jitter (global-fallback call site) (#12695)
Per Diego's review feedback on the original PR (which was closed for base drift + over-broad scope):
- Helper in tests/unit/ (not open-sse/__tests__) - 7 node:test cases
- Drop 429 from retryable set (keep 502/503/504) - 429 must respect the
RateLimit-Reset header and be surfaced, not silently retried
- Touch ONLY the global-fallback call site in chat.ts - the
combo-target loop has its own retry semantics already
The wrap uses decorrelated full-jitter exponential backoff (per AWS
retry guidance), abort-aware (client disconnect cancels immediately),
and respects the AbortSignal via a sleep that throws on abort.
Re-cherry-pick: dropped all unrelated drift from the original PR
(videoBridgeLog plumbing, forcedConnectionId refactor,
comboCheckProvider/ghComboGate, getPassthroughProviders, Moonshot
quota fetcher, withSelectedConnectionHeader, classifyProviderBreakerResult,
reanchorVideoBridgeRedaction, etc.). chat.ts is now +20/-16 — purely the
import + the wrap around handleSingleModelChat in the global-fallback
block. Diego should now see a clean, focused diff.
Fixes#12695
* fix(sse): address kilo-code-bot review on transientBackendRetry helper
Per the review comment on the previous commit:
- Jitter formula now matches documented AWS decorrelated jitter
(temp = min(cap, random(base, prev*3))) instead of an ad-hoc
baseMs + rand*prev*2 approximation
- AbortSignal is honoured by the default sleep implementation (was
previously only honoured by the loop's pre-attempt check)
- 'source' option added to TransientRetryOptions so callers can
propagate an observability label (e.g. 'global-fallback') through
onRetry
Adds 3 tests:
- default sleep respects AbortSignal without custom sleep
- onRetry receives source label
- decorrelated jitter is bounded by capMs
10/10 unit tests pass.
---------
Co-authored-by: KooshaPari <kooshapari@users.noreply.github.com>
* fix(compression): log warnings for unreadable settings rows
getCompressionSettings() silently skipped non-string (BLOB) and
invalid-JSON settings rows, making it impossible to diagnose config
drift between the panel and the runtime.
Now logs a warn-level message for each unreadable row, including the
key name and a remediation hint (re-save from the Storage panel).
Also warns when the 'engines' row exists but yields no valid toggles,
so operators know their panel-configured engines map is being silently
replaced by the legacy fallback.
Fixes#13456
* test(compression): cover getCompressionSettings warnings for unreadable rows
The test for #13456 only asserted a stubbed console.warn recorded a
message and never called getCompressionSettings(), so it never
exercised the production change. Seed a BLOB row, an invalid-JSON row,
and an 'engines' row that isn't a usable object, and assert the
resulting warnings; also assert a legitimately empty (but valid)
'engines' map does not warn.
Also stop warning on a valid-but-empty 'engines' row: parseStoredEnginesMap
returns null both for an unreadable row and for a well-formed {} (an
operator who deliberately disabled every engine), so only warn when the
stored value isn't a usable object at all.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: Koosha Pari <koosha@phenotype.ai>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
