mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-02 05:12:11 +03:00
9dcbbd18c8d707218d1565a79bdc29a0490c6502
2448 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
ikelvingo
|
9dcbbd18c8 |
fix(i18n): restore brand proper nouns and unify terminology in zh-CN and zh-TW (#8355)
* chore(ci): add .mergify.yml to main — Mergify only reads config from the default branch (#7168) * fix(ci): add the auto-enqueue pull_request_rule to the Mergify config (queue_conditions alone are eligibility-only) (#7179) * fix(ci): migrate Mergify auto-enqueue to merge_protections_settings.auto_merge_conditions (rules-based path is EOL 2026-07-16) (#7216) * fix(ci): drop Mergify batch settings (batching is a paid-tier feature; free plan queue is serial) (#7220) * fix(ci): merge queue tolerates the advisory dast-smoke failure (its GH-hosted build hang dequeued every attempt) (#7225) * test(ci): make the #6634 selfref guard hermetic — main's copy hard-fails every PR (#7341) main's copy of this test still does git I/O inside a unit test: const baseSrc = git(['show', 'origin/main:' + FILE]); Runners check out a shallow single ref, so origin/main does not resolve and the test dies with 'fatal: invalid object name origin/main'. Every PR into main fails Unit Tests (7/8) on it — today that is #7313, #7315, #7316, #7334, #7336 and #7337, six PRs red on a defect none of them introduced. #7313 has no other red at all. release/v3.8.49 already carries a fix ( |
||
|
Jay Ongg
|
58ab8b1d2c |
Clicking a provider card hitting back loses scroll position (#8349)
* clicking a provider card hitting back loses scroll position * clean up code * Rename some funcitons * minor UX updates * add null-guard in highlight() * Add providerCardHandle tests * add highlight tests. refactored code into separate utility functions * Minor change to skip a firstElementChild call - use a ref to access the Link inside ProviderCard. |
||
|
Adrian Rogala
|
9a0e764459 |
feat(cli): replace ANTHROPIC_SMALL_FAST_MODEL with Fable default (#8343)
Claude Code retired ANTHROPIC_SMALL_FAST_MODEL; expose ANTHROPIC_DEFAULT_FABLE_MODEL from the claude registry instead. |
||
|
Diego Rodrigues de Sa e Souza
|
53a91b3df8 |
feat(api): quota-aware fallback routing for web-fetch providers (#8297) (#8335)
Mirror the search/route.ts pattern for /v1/web/fetch: skip rate-limited stubs instead of letting them short-circuit auto-select, walk the fixed-priority pool (fill-first) with a request-time fallback on retryable/quota upstream statuses (429 always; 402/403 for Firecrawl/Tavily/TinyFish quota-style tiers), and return a proper 429 (with Retry-After) when the whole pool is exhausted instead of a generic 400. Explicit-provider requests never silently fall back. |
||
|
Diego Rodrigues de Sa e Souza
|
b8901b6506 | feat(db): persist caller session tag into call_logs for per-session cost attribution (#8249) (#8334) | ||
|
Diego Rodrigues de Sa e Souza
|
1cafd328c7 | fix(dashboard): persist compression engine detail settings (Headroom / session dedup / CCR) instead of dropping them on save (#8388) (#8404) | ||
|
Diego Rodrigues de Sa e Souza
|
544ae2d3da | fix(api): accept a missing status query-param on GET /api/plugins instead of rejecting null with Invalid status value (#8374) (#8399) | ||
|
Diego Rodrigues de Sa e Souza
|
312e24e785 |
fix(plugins): fire registered+active plugin hooks (onRequest/onResponse/onError) during proxying instead of never invoking them (#8395) (#8449)
Co-authored-by: ikelvingo <im.kelvinwong@gmail.com> |
||
|
Diego Rodrigues de Sa e Souza
|
d7f9475864 |
fix(backend): make disabling the global per-key proxy toggle override existing per-key proxy assignments (#8385) (#8447)
Co-authored-by: ikelvingo <im.kelvinwong@gmail.com> |
||
|
Diego Rodrigues de Sa e Souza
|
9a78ea2225 |
fix(providers): stop marking a multi-quota-window provider exhausted when only some windows are depleted (LIMIT-200 snapshot eviction drops idle healthy windows) (#8431) (#8445)
Co-authored-by: ikelvingo <im.kelvinwong@gmail.com> |
||
|
Diego Rodrigues de Sa e Souza
|
9b7bd6e5d3 | fix(api): stop leaking the internal provider UUID in /v1/models and honor the configured prefix (#8327) (#8361) | ||
|
Diego Rodrigues de Sa e Souza
|
fa46d93941 | fix(api): accept the current compatible-provider connection id scheme in the models test route (#8326) (#8359) | ||
|
Diego Rodrigues de Sa e Souza
|
493708f575 | fix(sse): strip third-party-agent signals from the Hermes system prompt that trigger Anthropic 400 extra-usage (#8350) (#8358) | ||
|
Diego Rodrigues de Sa e Souza
|
70275be59b | fix(dashboard): show custom provider_nodes providers in the Topology view instead of only AI_PROVIDERS (#8328) (#8357) | ||
|
Markus Hartung
|
dec4a67fe7 |
fix(memory): self-heal upsertVector/deleteVector from a raced vec_memories drop (#8337)
Live incident: memory.vec.upsert.fail {"error":"no such table: vec_memories"}
recurred repeatedly right after restarts, even though ensureReady() is called
immediately beforehand. ensureReady()'s signature-check-then-maybe-recreate
logic (resetForSignature does DROP TABLE IF EXISTS + CREATE VIRTUAL TABLE) is
not synchronized against a concurrent caller's upsertVector/deleteVector -- a
second in-flight memory write that independently decides (from a stale read
of memory_vec_meta) it also needs to reset the table can drop it out from
under another write's insert. Confirmed live: memory_vec_meta showed
vec_loaded=0 for the entire session across many restarts, then flipped to 1
mid-investigation once one attempt finally completed without interruption --
consistent with an intermittent race, not a permanently broken path (verified
the underlying sqlite-vec extension and CREATE VIRTUAL TABLE statement work
correctly in isolation, both on the host and inside the production container).
Rather than chase the exact interleaving (every underlying SQLite call is
synchronous via better-sqlite3, so the race window is narrow and did not
reproduce under simple Promise.all stress tests), makes the write path
resilient to arriving after the table was dropped: on a "no such table"
error, recreate vec_memories from the last-known-good memory_vec_meta
dimension and retry once.
Co-authored-by: Markus Hartung <markus.hartream@gmail.com>
|
||
|
Ravi Tharuma
|
07067841b5 |
feat(combo): enforce provider and model family invariants (#8304)
* feat(combo): enforce provider and model family invariants Closes #8279 Co-Authored-By: Ravi Tharuma <ravitharuma@users.noreply.github.com> * fix(combo): complete invariant enforcement paths Map invariant failures to structured API errors, correct target diagnostics, and validate restored combos inside the existing migration transaction. Co-Authored-By: Ravi Tharuma <noreply@github.com> --------- Co-authored-by: Ravi Tharuma <RaviTharuma@users.noreply.github.com> Co-authored-by: Ravi Tharuma <noreply@github.com> Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> |
||
|
Bob.Hou
|
97d948cc9e |
fix(antigravity): add missing gemini-3.6-flash pricing rows to ag OAuth pricing (#8290)
release/v3.8.49 already ships the Gemini 3.6 Flash catalog entries (AGY_PUBLIC_MODELS, ANTIGRAVITY_PUBLIC_MODELS, MODEL_SPECS with supportsThinking: false — Antigravity still rejects client-supplied thinking params) via #8013. What was still missing: the `ag` pricing rows in DEFAULT_PRICING_OAUTH, so getPricingForModel("ag", id) returned null for the three tiers and cost/quota calculations silently fell back to $0. Pricing: $1.50 input / $7.50 output / $0.15 cached per MTok (Google's 2026-07-21 announcement), matching the existing 3.5-flash schedule shape. Thinking tokens billed at output rate. Extends the existing pricing-ag-flash-tiers.test.ts (RED-first: all three tiers failed the "non-null pricing row" assertion before this change) rather than re-adding the already-shipped catalog/modelSpecs entries. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> |
||
|
Prudhvi Vuda
|
af60f41e99 |
fix(providers): reconcile Kimi K3 vision when attachment contradicts modalities (#8250) (#8313)
* fix(providers): reconcile Kimi K3 vision when attachment contradicts modalities Synced models.dev rows for kimi-coding*/k3 can ship attachment=false while modalities_input still lists image/video. Prefer the modality signal (and normalize at sync + resolve) so supportsVision, attachment, and exposed modalities agree. Closes #8250 * fix(providers): keep Kimi K3 static fallback text-only (#8250) The Kimi K3 vision reconciliation added supportsVision=true directly to the kimi-coding registry entry for id "k3". That entry is the static/stable fallback catalog used when discovered capabilities are unavailable, and it must stay text-only per #4071 — the vision fix is already applied correctly on the discovered path via MODEL_SPECS["kimi-k3"] (aliases: ["k3"]) and modelCapabilities.ts::resolveVisionCapability. Restores the invariants guarded by tests/unit/kimi-k2.7-code-registration.test.ts ("Kimi Code k3 fallback leaves discovered capabilities unset") and tests/unit/catalog-updates-v3829-kimi-qwen.test.ts ("kmca stable fallback only carries documented static capabilities"). Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> --------- Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> |
||
|
Bob.Hou
|
406f41de30 |
fix(translator): cap thinking budget on explicit budget_tokens path (#8312)
* fix(translator): cap thinking budget on explicit budget_tokens path * fix(translator): stop dropping thinkingConfig on cap-0 reasoning_effort path The thinking-budget-cap guard added in this branch skipped thinkingConfig entirely whenever a model's thinkingBudgetCap was 0 (e.g. gemini-3-flash), including on the reasoning_effort/budgetMap path. That regressed the pre-#6943 native-defaults contract (thinkingBudget 0 / includeThoughts false must still be present) and crashed callers that read `.thinkingConfig.thinkingBudget` unconditionally (translator-openai-to-gemini-defaults.test.ts). Also restore includeThoughts:true on the Claude-format explicit thinking.budget_tokens path (openai-to-gemini.ts's Claude-format field and claude-to-gemini.ts's native thinking field): budget_tokens:0 there is the client's dynamic-thinking sentinel (#6813), not an off-switch, and must stay true even after the new capping — the cap must only clamp positive explicit values, never flip the zero sentinel's semantics. Updates two tests this branch added that encoded the incorrect "omit thinkingConfig / includeThoughts:false for the 0 sentinel" behavior, to match the pre-existing, still-required contracts above. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> * fix(providers): revert scope-creep flip of Gemini 3.5/3.6 Flash supportsThinking The thinking-budget-cap fix accidentally expanded 5 shorthand modelSpecs entries (gemini-3.5-flash, gemini-3.5-flash-low, gemini-3.6-flash-high/ medium/low) into explicit objects setting supportsThinking:true and thinkingBudgetCap:24576. That flip was unrelated to the two proven test regressions (translator-openai-to-gemini-defaults.test.ts and claude-to-gemini-budget-tokens-zero-6813.test.ts, which only exercise gemini-3-flash-preview, gemini-3.1-pro and gemini-2.5-pro) and reopens a deliberately closed path from #8013: Antigravity still rejects client-supplied thinking params for these Gemini 3.5/3.6 Flash tier ids, so supportsThinking must stay false (inherited from GEMINI_35_FLASH_MODEL_SPEC). Reverted all 5 entries back to the release shorthand `{ ...GEMINI_35_FLASH_MODEL_SPEC }`. gemini-3-flash, gemini-3.1-pro and gemini-2.5-pro (the models the regression tests actually exercise) were already correctly specced in the release baseline and are untouched. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> --------- Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> |
||
|
Diego Rodrigues de Sa e Souza
|
afe3a931f9 |
fix(i18n): restore #8219 CacheSettingsTab key sync + synthetic fixture for zh-TW repro test (#8387)
Root cause (two independent causes):
1. PR #8219 (commit
|
||
|
Diego Rodrigues de Sa e Souza
|
c5c27b813a |
fix(sse): cap exact cooldowns only when synthetic — verified upstream resets pass uncapped (#8393)
Contract vs cap: #6863 requires a model lockout to honor a VERIFIED upstream quota reset exactly (e.g. Antigravity "Resets in 92h27m28s", shipped in v3.8.47). #7940 requires SYNTHETIC exact-cooldown estimates (the quota_exhausted until-midnight heuristic) to respect the operator's maxCooldownMs so they cannot balloon unbounded. Both are legitimate, non-conflicting contracts — they apply to different kinds of values. Root cause: #7980 (fixing #7940) changed recordModelLockoutFailure() in open-sse/services/accountFallback.ts to unconditionally clamp every exactCooldownMs against maxCooldownMs, with no way to distinguish a verified upstream reset from a synthetic estimate. A real ~92h reset got clamped to the operator's ~30min cap, and the router went on hammering 429 against quota that was known not to recover for days — regressing #6863's contract by omission, not by new policy (the "honor it exactly" docstrings on selectLockoutCooldownMs() and its call sites were left untouched and now describe dead code). Fix: add an opt-in `exactCooldownVerified` flag to recordModelLockoutFailure()'s options. When true, exactCooldownMs bypasses the maxCooldownMs clamp entirely; when false/omitted (the default), behavior is byte-identical to before this change. Set the flag only at the 4 call sites that already carry upstream provenance for the value they pass — usedUpstreamRetryHint / quotaResetHintMs from checkFallbackError(): - open-sse/services/combo.ts (2 sites): exactCooldownVerified mirrors lockoutHintMs > 0, which is only ever nonzero when it traces back to a genuine upstream signal. - src/sse/services/auth.ts (2 sites): exactCooldownVerified mirrors the same usedUpstreamRetryHint / quotaResetHintMs check already used to derive exactCooldownMs at each site. The quota_exhausted → until-midnight synthetic default and plain exponential backoff are untouched and stay capped, per #7940. The two other recordModelLockoutFailure call sites (combo.ts quality failure, auth.ts local-404/grok-web-403) never carry a verified hint and were left unmodified. Validation (TDD): tests/unit/combo-lockout-quota-reset-6863.test.ts red→green with its assertions unchanged (was clamping ~332,848,000ms to ~1,799,995ms; now honors the parsed reset). Added a boundary pair to tests/unit/model-lockout-exact-cooldown-cap.test.ts proving the same magnitude resolves differently by provenance: synthetic stays capped, verified passes through whole. Full existing suite in that file plus combo-model-lockout-honors-reset-1308.test.ts stay green unmodified. Swept 45 lockout/cooldown-adjacent test files (502/505 passing); the 3 failures reproduce byte-identical on a pristine origin/release/v3.8.49 checkout (PROVIDER_BREAKER_FAILURE_STATUSES ReferenceError in untouched chat.ts, and a documented timing-sensitive serial test) — confirmed pre-existing, out of this fix's scope. npm run typecheck:core and npm run lint are clean. Refs #6863 Refs #7940 Refs #7980 |
||
|
Diego Rodrigues de Sa e Souza
|
f0096f0224 |
fix(resilience): terminal-skip spares the recoverable GitHub Copilot no_refresh_token state (#8389)
Cause: #8182's terminal-connection guard in checkConnection() returns early for any testStatus in {credits_exhausted, banned, expired} to stop the sweep from wasting CPU/network probing connections that can never self-heal. But testStatus="expired" + errorCode="no_refresh_token" is exactly the state the pre-existing GitHub Copilot self-heal targets (isGitHubAccessTokenOnlyConnection + canClearGitHubNoRefreshTokenState, ~line 83-97 / 413-492): a Copilot connection with no OAuth refresh token but a still-valid copilotToken, which the sweep is supposed to flip back to "active". With the new guard placed ahead of that block unconditionally, the self-heal became unreachable for exactly the state it exists to clear. Impact: healthy GitHub Copilot connections that once lost their OAuth refresh token got stuck at testStatus="expired" in the dashboard forever, even though their Copilot sub-token kept working and the sweep would have cleared the stale status back to "active" every cycle before #8182. Fix: carve out the exact recoverable shape from the terminal-skip guard — testStatus==="expired" && errorCode==="no_refresh_token" && isGitHubAccessTokenOnlyConnection(conn) — so the guard still skips every other terminal case (credits_exhausted, banned, and "expired" for any other reason) untouched, matching #8182's original intent. Validation: tests/unit/token-health-no-refresh-token-expired-5326.test.ts was red (1 fail / 4 pass) before the fix — "checkConnection clears stale no_refresh_token state for usable GitHub Copilot connections" asserted testStatus flips back to "active" but got "expired". Green after the fix (6/6, including a new boundary test proving a GitHub Copilot connection expired for any OTHER reason, e.g. errorCode "invalid_grant", is still skipped untouched). Also reran the adjacent checkConnection/tokenHealthCheck suites (token-health-check.test.ts, token-health-check-circuit-breaker.test.ts, apikey-connection-health-check.test.ts, token-health-check-sweep.test.ts, token-health-check-tickms-defined.test.ts, tokenHealthCheck-batchSize.test.ts, codex-oauth-refresh-persist-6352.test.ts, oauth-providers-error-handling.test.ts) — all green, confirming #8182's terminal-skip behavior is otherwise unchanged. npm run typecheck:core clean. Refs #8182 Refs #8286 Refs #5326 |
||
|
Diego Rodrigues de Sa e Souza
|
3b4f4afc9d |
fix(sse): re-export PROVIDER_BREAKER_FAILURE_STATUSES for the orphaned all-rate-limited breaker path (#8390)
Root cause: #8013 extracted shouldTripProviderBreakerForResult() from src/sse/handlers/chat.ts into the new src/sse/handlers/chatPredicates.ts, taking the (non-exported) const PROVIDER_BREAKER_FAILURE_STATUSES with it. A second, independent use of that const survived in chat.ts's handleSingleModelChat(), in the "all credentials rate-limited" block (~line 1340) — that reference was left orphaned by the extraction. Production impact: any request where every credential for a provider+model is simultaneously rate-limited throws `ReferenceError: PROVIDER_BREAKER_FAILURE_STATUSES is not defined` at runtime in that code path. Concretely this meant: - breaker._onFailure() was unreachable on the all-rate-limited path, so the provider circuit breaker could not trip from it - the ReferenceError propagated up and got mapped to a generic 502, masking the real 503 upstream-unavailable status in combo responses - the issue-agent route surfaced a generic 400 instead of the actual 429 provider-rate-limited response Fix: export PROVIDER_BREAKER_FAILURE_STATUSES from chatPredicates.ts and add it to chat.ts's existing import block from that module. No behavior change — the classification set ([408, 500, 502, 503, 504]) is unchanged, this only repairs the broken reference. Also re-points tests/unit/nvidia-quota-phase1.test.ts's regex-based declaration check at chatPredicates.ts, where the const now actually lives (it previously read chat.ts via fs+regex and silently failed to find the declaration). The regex and the classification assertions themselves are unchanged — this test still proves 429 is excluded from the whole-provider breaker. Refs #8013 |
||
|
Diego Rodrigues de Sa e Souza
|
3504050fcf | fix(providers): route noauth opencode-zen connections through their assigned proxy (#8324) | ||
|
Diego Rodrigues de Sa e Souza
|
35541c06cd | fix(providers): carve cookie-auth providers out of terminal 401 'expired' classification so one 401 cooldowns instead of killing the connection (#8321) | ||
|
Ravi Tharuma
|
8a2a3d48b0 |
perf(api): singleflight version lookups (#8278) (#8301)
Co-authored-by: Ravi Tharuma <RaviTharuma@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> |
||
|
backryun
|
b84f86ad4f | fix(runtime): isolate unique 8177 repairs (#8298) | ||
|
Ravi Tharuma
|
08a21bcf27 |
fix(backend): add structure-aware chat admission (#8296)
Co-authored-by: Ravi Tharuma <RaviTharuma@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Sean Ford
|
14468cdec5 |
fix(bifrost): send v-prefixed transport version, not bare semver (#8194)
* fix(bifrost): send v-prefixed transport version, not bare semver bifrost.ts resolveSpawnArgs() set BIFROST_TRANSPORT_VERSION straight from getInstalledVersionSync(), which reads the raw "version" field out of node_modules/@maximhq/bifrost/package.json - always bare semver per npm convention (e.g. "1.6.3"). @maximhq/bifrost's own bin.js validates that env var against /^v\d+\.\d+\.\d+(?:-[0-9A-Za-z.-]+)?$/ or the literal "latest" and rejects anything else with "Invalid transport version format", exiting immediately. Every embedded Bifrost instance failed to start as a result. Adds formatTransportVersion() to normalize at the call site that owns the env var, so bifrost.ts and the upstream @maximhq/bifrost package both stay exactly as designed - no changes needed to bifrost itself. Strengthens the existing resolveSpawnArgs test to assert the actual v-prefix format (it previously only checked for a non-empty string, the same gap #6877 called out for cliproxy's pre-existing test), and adds a dedicated regression test file covering the pure formatTransportVersion() helper plus a real-filesystem resolveSpawnArgs() integration check. Found and fixed while self-hosting OmniRoute and diagnosing why bifrost kept crash-looping on startup. * fix(bifrost): resolve install dir lazily so version read honors runtime DATA_DIR Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> --------- Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> Co-authored-by: seanford <seanford@users.noreply.github.com> |
||
|
growab
|
e9f297021e |
fix(usage): correct token/request counting for 30D/90D/YTD/ALL ranges (#7300)
* chore(ci): add .mergify.yml to main — Mergify only reads config from the default branch (#7168)
* fix(ci): add the auto-enqueue pull_request_rule to the Mergify config (queue_conditions alone are eligibility-only) (#7179)
* fix(ci): migrate Mergify auto-enqueue to merge_protections_settings.auto_merge_conditions (rules-based path is EOL 2026-07-16) (#7216)
* fix(ci): drop Mergify batch settings (batching is a paid-tier feature; free plan queue is serial) (#7220)
* fix(ci): merge queue tolerates the advisory dast-smoke failure (its GH-hosted build hang dequeued every attempt) (#7225)
* fix(usage): correct token/request counting for 30D/90D/YTD/ALL ranges
Two bugs caused incorrect usage statistics for date ranges beyond the
raw data retention window:
1. Cutoff mismatch: the analytics route computed rawCutoffDate from
aggregation.rawDataRetentionDays (migration 046 seeds =7) while
cleanupUsageHistory rolls up and deletes at retention.usageHistory
(=30). The window [day-30, day-7) existed in usage_history but was
excluded from BOTH UNION legs — raw leg floored at day-7, aggregated
leg ended at day-7 — producing undercounted token sums for 30D,
90D, YTD, and ALL ranges.
Fix: use dbSettings.retention.usageHistory for the raw cutoff in
both route.ts and getRawDataCutoffDate() (aggregateHistory.ts),
matching the actual cleanup boundary.
2. Request undercount: COUNT(*) on the unified source counted each
daily_usage_summary row as 1, not total_requests. A day with 50
rolled-up requests counted as 1.
Fix: add a 'requests' column to both UNION legs (raw: 1, aggregated:
total_requests), change COUNT(*) to SUM(requests) in 6 query
functions, and change successfulRequests from
SUM(CASE WHEN success=1 THEN 1 ELSE 0 END) to
SUM(CASE WHEN success=1 THEN requests ELSE 0 END). Also set
agg leg latency_ms to NULL so AVG(latency_ms) is not skewed.
Tests: 33/33 source-level tests pass (db-usageanalytics-split.test.ts),
verifying 'requests' column presence and SUM(requests) usage in all
affected queries. DB-level integration test added to
usage-analytics.test.ts (requires node + better-sqlite3).
* fix(ci): green CI reds on #7300 — file-size ratchet, stale test fixture, shallow-checkout selfref test
- src/app/api/usage/analytics/route.ts: trim the new comment to keep the file
at the frozen file-size baseline (942 lines) after the retention.usageHistory
cutoff fix — no logic change.
- tests/unit/usage-analytics-route.test.ts: the pre-existing "does not
double-count raw and aggregated rows" test hardcoded a 30-day cutoff that
matched the OLD (buggy) aggregation.rawDataRetentionDays default. Now that
the raw/aggregated boundary correctly uses retention.usageHistory (365 days
by default, matching cleanupUsageHistory's actual rollup/delete boundary),
the fixture's synthetic "old" row was within the raw window and got
excluded from the aggregated leg. Read the real retention setting instead
of hardcoding 30 so the fixture reflects the corrected boundary. Same
assertions (still expects no double-counting, totalRequests=2,
totalTokens=185) — only the fixture dates change.
- tests/unit/check-test-masking-selfref-6634.test.ts: tolerate the shallow/
single-ref checkout used by GitHub-hosted Unit Tests runners (no local
origin/main ref) by fetching it on demand and skipping (never failing) when
unreachable offline. Matches the fix already applied on another branch
(2e42b8efc/#7174) for the same root cause, not yet on main.
Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
* test(ci): make the #6634 selfref test checkout-independent (read the real file, no git ref)
The previous on-demand `git fetch origin main` + t.skip() fallback cleared the
shallow-checkout failure but tripped the PR Test Policy's test-masking gate
(a new .skip counts as a silenced assert — correctly so).
Drop the git dependency entirely instead: read the REAL current source of
tests/unit/check-test-masking.test.ts from disk (so the actual #6404 fixture
literals stay under test) and model the pre-#6404 state with an empty base,
which maximizes headTaut - baseTaut — the strictest input for the exclusion
this test asserts. No skip, no weakened assertion, same deepEqual guarantee.
Verified non-vacuous: neutralizing SELF_TEST_FIXTURE_RE in
scripts/check/check-test-masking.mjs makes this test fail; restoring it makes
it pass.
Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
* test(ci): make the #6634 selfref guard hermetic — main's copy hard-fails every PR (#7341)
main's copy of this test still does git I/O inside a unit test:
const baseSrc = git(['show', 'origin/main:' + FILE]);
Runners check out a shallow single ref, so origin/main does not resolve and the
test dies with 'fatal: invalid object name origin/main'. Every PR into main
fails Unit Tests (7/8) on it — today that is #7313, #7315, #7316, #7334, #7336
and #7337, six PRs red on a defect none of them introduced. #7313 has no other
red at all.
release/v3.8.49 already carries a fix (
|
||
|
backryun
|
869d08ff8b |
Add Alibaba-family media model support (#8266)
* Add Alibaba-family media models * chore(quality): rebaseline imageRegistry+cognitive for #8266 media own-growth --------- Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> |
||
|
Alberto Punter
|
2670674107 | fix(i18n): re-sync and complete es-ES translations with latest release/v3.8.49 (#8289) | ||
|
Ravi Tharuma
|
069d5a7925 |
fix(dashboard): stop sidebar RSC prefetch storms (#8292)
* fix(dashboard): disable sidebar route prefetch Co-Authored-By: Claude <noreply@anthropic.com> * test(dashboard): cover sidebar prefetch traffic Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Ravi Tharuma <RaviTharuma@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Austin Liu
|
99ad15a37f |
fix(resilience): skip terminal connections in token health check sweep (#8182) (#8286)
Background token health check sweep was probing connections with terminal statuses (credits_exhausted / banned / expired) on every cycle, wasting CPU and network. These connections can never self-heal via token refresh — they need manual re-auth or credit top-up. Add a terminal status guard in checkConnection() that mirrors the existing isTerminalConnectionStatus() in auth.ts and TERMINAL_CONNECTION_STATUSES in connectionRecovery.ts. Verified by reporter: after manually disabling 11 credits_exhausted connections, CPU dropped from ~53% to ~2.7%. This fix automates that skip so the sweep never touches terminal connections in the first place. Co-authored-by: Austin Liu <austinliu@Austins-MacBook-Air-3.local> |
||
|
Kayn Xu
|
56e2d2efb0 |
fix(providers): update Learn more documentation link (#8284)
* fix(providers): update Learn more documentation link * docs(changelog): note provider documentation link fix |
||
|
backryun
|
2f65339378 |
fix(providers): limit Gemini CLI to legacy OAuth refresh (#8275)
#8232 correctly targeted OAuth auto-refresh for legacy stored Gemini CLI connections, but exceeded that compatibility goal by recreating a complete routable and UI-visible provider with an Antigravity model catalog. Preserve legacy refresh by mapping gemini-cli rows to the existing Gemini OAuth credentials. Remove the public provider registry, OAuth preset, model routing snapshot, and canonical-provider tests while keeping Gemini API and Antigravity unchanged. |
||
|
nguyenha935
|
a6eb4d8166 |
fix: normalize Codex URLs and dashboard regressions (#8233)
Co-authored-by: nguyenha935 <208228297+nguyenha935@users.noreply.github.com> Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> |
||
|
Apostol Apostolov
|
f3ed4a49d4 |
feat(providers): add weekly quota tracking for grok-web (#8127)
* feat(providers): add weekly quota fetcher for grok-web (grok.com SSO) Implements a bespoke QuotaFetcher for the grok-web provider that: - Reads OIDC tokens from ~/.grok/auth.json (local Grok CLI login) - Refreshes tokens via auth.x.ai OIDC if expired - Calls https://cli-chat-proxy.grok.com/v1/billing?format=credits - Returns a single 'weekly' window with creditUsagePercent and resetAt - Caches results with 60s TTL (matching codexQuotaFetcher pattern) - Supports GROK_AUTH_PATH env var override for testing - Registers in chat.ts before registerGenericQuotaFetchers Tests cover: missing auth, successful fetch with header verification, 401-triggered token refresh with retry, 60s cache TTL, and preflight integration. Closes #6444 * chore(quality): rebaseline chat.ts for #8127 own-growth --------- Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> |
||
|
Paijo
|
2a865aaaa7 |
feat(settings): configurable model catalog cache TTL (#8219)
* feat(settings): configurable model catalog cache TTL Add modelCatalogCacheTtlMs to DatabaseSettings with default 1500ms. Extend cache-config API route to accept the new field. Replace hardcoded catalog cache TTL with dynamic settings value. Add 'Cache' settings tab at /dashboard/settings/cache with sidebar entry, i18n keys, header description, and legacy route redirect. * feat(combo): add model connection filter toggle to ModelSelectModal Adds a 'Show configured only' checkbox below the search bar in ModelSelectModal that filters each provider group's models through hasEligibleConnectionForModel. Toggle state persists in localStorage. - Import hasEligibleConnectionForModel from domain/connectionModelRules - showConfiguredOnly state + localStorage persistence - connectionFilteredGroups memo layered on filteredGroups - Renders both provider section and empty state from connectionFilteredGroups * test(combo): add connection filter toggle tests for ModelSelectModal Three test cases: (1) hide excluded models when toggle on, (2) show empty state when all models excluded, (3) drop provider group when all its models excluded. All 3 tests pass. * fix(settings): correct cache-config route import + add route/tab coverage The cache-config route imported get/update helpers from a nonexistent module (@/lib/localDb/databaseSettings) and called an undefined updateSettings() in PUT, crashing every request. Import the real databaseSettings module (matching the sibling database/route.ts convention) and call updateDatabaseSettings(); idempotencyWindowMs is routed through the flat @/lib/db/settings module instead, since that is where it is actually read at runtime (idempotencyLayer.ts, runtimeSettings.ts) — it was never part of the databaseSettings "cache" section type. Also fixes a dashboard-typecheck regression in ModelSelectModal.tsx: the new connection-filter toggle called hasEligibleConnectionForModel() with activeProviders entries typed too narrowly to include providerSpecificData, which real connection objects carry at runtime. Adds: - tests/unit/cache-config-route-8219.test.ts: GET/PUT resolve without crashing, modelCatalogCacheTtlMs and idempotencyWindowMs round-trip. - tests/unit/ui/cache-settings-tab-bounds-8219.test.tsx: CacheSettingsTab min/max TTL bounds (100ms/60000ms) gate the Save button and surface a validation message; in-bounds values PUT correctly. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> * chore(quality): rebaseline sections.ts for #8219 own-growth --------- Co-authored-by: oyi77 <oyi77@users.noreply.github.com> Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> |
||
|
Álvaro Ángel Molina
|
81ade9e122 |
feat(providers): add Typhoon (Thailand) and Inception Mercury diffusion LLM (#8170)
* feat(providers): add Typhoon and Inception Mercury API-key providers Two OpenAI-compatible API-key providers, each verified against a live endpoint smoke test with a negative control before registration: an unknown route answers 404 while /v1/chat/completions answers 401, which rules out gateways that reply identically to every path. - typhoon (SCB 10X, Thailand): first Thai-first provider in the catalog. /v1/models answers 200 unauthenticated and serves exactly one chat model, typhoon-v2.5-30b-a3b-instruct (128K ctx). The docs also list typhoon-v2.1-12b-instruct, but the live endpoint no longer serves it, so it is deliberately not registered. The typhoon-ocr* and typhoon-asr* entries are OCR and speech models, not chat, and are omitted. - inception (Inception Labs): first diffusion LLM (dLLM) in the catalog. mercury-2 has a 128K context, 50K max output, and supports tools, json_mode and structured outputs. The mercury-coder models advertised on the vendor blog are no longer served by the live endpoint and are therefore not registered. Both expose a working /v1/models catalog, so they are added to NAMED_OPENAI_STYLE_PROVIDERS for discovery and key validation. Free-tier metadata is claimed only where it is documented and durable: Typhoon issues a free API key rate-limited to 5 req/s and 200 req/m, and Inception grants 10M tokens on signup with no card, so both are registered with hasFree: true. Provider count moves from 280 to 282; README/AGENTS/CLAUDE counters were stale at 278 and are resynced against docs/reference/PROVIDER_REFERENCE.md. * fix(8170): union frontier-labs Inception+Writer, regen ref+golden * fix(8170): close inception object + regen ref/golden --------- Co-authored-by: Álvaro Ángel Molina <alvaretto@users.noreply.github.com> Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> |
||
|
Álvaro Ángel Molina
|
7ca821aeee |
feat(providers): add Sarvam AI, Writer Palmyra and PLaMo API-key providers (#8161)
* feat(providers): add Sarvam AI, Writer Palmyra and PLaMo API-key providers Three OpenAI-compatible API-key providers, each verified against a live endpoint smoke test before registration: - sarvam (India): /v1/models answers 200 unauthenticated and lists sarvam-105b (128K ctx) and sarvam-30b (64K ctx). The older sarvam-m is discontinued upstream and is deliberately not registered. - writer (Palmyra): api.writer.com exposes the OpenAI alias /v1/chat/completions alongside its native /v1/chat — confirmed with a negative control, since an unknown route answers 404 'endpoint not available via API gateway' while /v1/chat/completions answers 401. Registers palmyra-x5 (1M ctx) and palmyra-x4 (128K ctx); the medical/financial/creative/vision variants are deprecated upstream and are omitted. - plamo (Preferred Networks, Japan): only plamo-3.0-prime (262K ctx) is registered. plamo-3.0-prime-beta is discontinued on 2026-07-31 and plamo-2.2-prime on 2026-09-30, so neither is worth wiring up. Free-tier metadata is claimed only where it is documented and durable: Sarvam ships a permanent signup credit, while PLaMo's 10M-token grant is a campaign that expires on 2026-07-31 and Writer documents no free tier, so both are registered with hasFree: false. * regen golden+ref --------- Co-authored-by: Álvaro Ángel Molina <alvaretto@users.noreply.github.com> Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> |
||
|
Álvaro Ángel Molina
|
6ba2260178 |
feat(providers): add CLOVA Studio, InternLM and Ant Ling API-key providers (#8077)
* feat(providers): add CLOVA Studio, InternLM and Ant Ling API-key providers Adds three OpenAI-compatible frontier-lab providers, closing regional gaps in the catalog (Korea had none; the Shanghai AI Lab and Ant Group families were both missing). - clova-studio: Naver HyperCLOVA X (HCX-007 reasoning, HCX-005 multimodal) on the current clovastudio.stream.ntruss.com host. The legacy clovastudio.apigw.ntruss.com endpoint is being deprecated and is not used. - internlm: Shanghai AI Lab Intern-S1 family (intern-s1-pro is a 1T MoE). Ships a free monthly quota, so it is flagged hasFree. - ant-ling: Ant Group / inclusionAI Ling-2.6-1T and Ring-2.6-1T. All three endpoints were smoke-tested: each returns HTTP 401 on <baseUrl>/models (endpoint live, awaiting auth) and resolves against public DNS. All three are registered for live model discovery, so their catalogs refresh from upstream. Known limitation: the ant-ling model ids are best-effort from public docs and are NOT verified against a live /v1/models response, which requires an API key. This is recorded in the registry comment and in the provider authHint so it is visible to operators rather than silently assumed. Its baseUrl is likewise not published in the public docs and was found by smoke test. Two AI SUTRA was evaluated for this batch and deliberately excluded: its documented endpoint api.two.ai does not resolve in public DNS (ENOTFOUND against 1.1.1.1 and 8.8.8.8, with www.two.ai resolving as control). The translate-path golden snapshot is regenerated; the change is additive only (209 -> 212 keys, exactly the three new providers, none removed or altered). * feat(providers): verify ant-ling against official docs, add Ling-2.6-flash and free tier The ant-ling entry was added with model ids marked best-effort because they could not be checked without an API key. Ant Ling's own documentation turns out to publish enough to verify them, so the uncertainty is now resolved: - The quickstart sample uses base_url "https://api.ant-ling.com/v1" with model "Ling-2.6-1T", confirming both the endpoint and the exact casing. - The pricing page bills exactly three models over the API, so Ling-2.6-flash was missing from the catalog and is added. - Each account gets 500,000 free tokens per day (resets 02:00 UTC+8, no rollover), so the provider is flagged hasFree with a freeNote. The Ming family (Ming-Flash-Omni, Ming-Light) is deliberately left out: it is documented as open-source / Ling Studio only and does not appear on the pricing page, so it is not served over this chat-completions API. That reasoning is recorded in the registry so it is not re-litigated later. The authHint no longer claims the ids are unverified, and now points at the API console (https://chat.ant-ling.com/open) where keys are actually created. Same correction applied to the en, pt-BR and vi message catalogs. * docs: sync provider counts to 283 and regenerate the provider reference --------- Co-authored-by: Álvaro Ángel Molina <alvaretto@users.noreply.github.com> Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> |
||
|
Rafael Dias Zendron
|
1e1941d551 |
fix(#8135): suppress sql.js build warning via non-analyzable dynamic import (#8184)
* fix(ci): resolve upstream-inherited check failures
* fix(build): suppress sql.js build warning via non-analyzable import
Replace literal import('sql.js') with a computed specifier and
webpackIgnore magic comment so Next.js/webpack doesn't try to
statically resolve sql.js/package.json during the build phase.
Fixes #8135
---------
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
|
||
|
Rafael Dias Zendron
|
9b2968fc07 |
fix(resilience): add max/step to NumberField for provider cooldown inputs (#8107) (#8203)
* fix(ci): resolve upstream-inherited check failures * fix(resilience): add max/step to NumberField for provider cooldown inputs Fixes #8107: integer input rejects typed value on Chrome/Windows because frontend allowed values exceeding backend zod schema limits. - Add and props to NumberField component - Apply correct limits for provider cooldown (min: 300000ms, max: 3600000ms) - Apply correct limits for waitForCooldown (maxRetries: 10, maxRetryWaitSec: 300) - Apply correct limits for requestQueue (requestsPerMinute: 1000, minTimeBetweenRequestsMs: 10000, concurrentRequests: 100, maxWaitMs: 300000, maxQueueDepth: 100000) - Apply correct limits for connection cooldown (baseCooldownMs: 3600000, maxBackoffSteps: 100) - Apply correct limits for provider breaker (failureThreshold: 1000, degradationThreshold: 1000, resetTimeoutMs: 300000) - Apply correct limits for combo cooldown (maxWaitMs: 30000, maxAttempts: 10, budgetMs: 300000) - Apply correct limits for quota share concurrency (enabled only - no numeric limits) * fix(resilience): clamp cooldown bounds before save + regression test (#8107) Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> --------- Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> |
||
|
小妍儿 ✨
|
b640b69078 |
feat(codex): support reference image edits (#8122)
* feat(codex): support reference image edits * docs(changelog): add Codex edit fragment * fix(codex): harden image edit admission * fix(security): redact image error credentials * fix(security): close error redaction bypasses * feat(codex): support multiple image references * fix(codex): preserve reference candidate semantics * chore(quality): rebaseline image-generation-handler.test.ts for #8122 codex image edits own-growth Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> --------- Co-authored-by: 千乘妍 (Xiaoyaner) <xiaoyaner0201@users.noreply.github.com> Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> |
||
|
Michael YC JO
|
f3d512eeff |
fix(dashboard): correct machine-translated Korean UI strings in ko.json (#8224)
Fix 527 mistranslated values in the Korean locale, all verified against
the en.json source:
- Restore protected product/protocol names garbled by machine translation
(응록→ngrok, 인류/인류학→Anthropic, 쌍둥이자리→Gemini, 반중력→Antigravity,
꼬리비늘 깔때기→Tailscale Funnel, 진공→VACUUM, 우편번호→ZIP)
- Fix wrong-sense homonym translations (달리기→실행 중 for Running,
장애인→비활성화됨 for Disabled, 열쇠→키 for Key, 안타→적중 for Hits,
유물→아티팩트 for Artifacts, 건강검진→상태 확인 for Healthcheck)
- Repair translated identifiers that broke literal values (양말5→socks5,
볼록-세션-id→convex-session-id, 채팅/완료→chat/completions,
메시지/보내기→message/send JSON-RPC methods)
- Replace key-name dumps shipped as values ("Table Name", "Overview
Title", "Cli Tools Redirect Title" etc.) with real Korean translations
- Unify ngrok casing (Ngrok→ngrok) and trailing punctuation with the
English source; align terminology across fixes (공급자, 폴백, 사용자 정의)
All {placeholder} tokens, markdown, and protected terms preserved
verbatim; i18n UI coverage and ko validation gates pass.
|
||
|
Prudhvi Vuda
|
a29341ff1b |
fix(memory): resolve remote embedding dimensions for reindex (#8074) (#8220)
Use getEmbeddingDimension() in resolveEmbeddingSource so sqlite-vec can create vec_memories before the first upsert, and abort reindex batches when ensureReady returns ready=false instead of wasting embed credits. |
||
|
Sean Ford
|
71887ae529 |
fix: restore OAuth auto-refresh for gemini-cli connections (#8232)
* fix: restore OAuth auto-refresh for gemini-cli connections
gemini-cli OAuth connections had no PROVIDERS registry entry at all, so
the token-refresh health check permanently skipped them once the access
token expired, forcing a full re-authentication instead of using the
still-valid refresh token.
Two layered gaps, both required:
1. supportsTokenRefresh()'s explicit allow-set had "gemini" but not
"gemini-cli" (the id actually stored on these connections), and its
PROVIDERS[e].tokenUrl fallback also failed since...
2. ...open-sse/config/providers registry had zero entry for "gemini-cli"
at all: no clientId/clientSecret/tokenUrl/refreshUrl, so even the
generic refresh path had nothing to refresh with.
Adds a "gemini-cli" registry entry mirroring antigravity's Google
Cloud Code OAuth shape, reusing the same well-known public Gemini CLI
client credentials already embedded (and already used, unchanged, by
the Gemini Studio API-key provider's own oauth block) via
resolvePublicCred("gemini_id"/"gemini_alt"). Adds "gemini-cli" to the
explicit refresh allow-set, the Google-refresh dispatch case, and the
15-minute non-rotating-token proactive lead alongside antigravity/agy.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix: register gemini-cli in canonical provider list (provider-consistency gate)
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(gemini-cli): use ANTIGRAVITY_RUNTIME_BASE_URLS (renamed by #8013 antigravity split)
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: seanford <seanford@users.noreply.github.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
|
||
|
Sean Ford
|
337373f8f0 |
fix(services): resolve and record a real pid when adopting a service (#8218)
ServiceSupervisor.start(), when probeBeforeSpawn detects an already-healthy instance on the target port, "adopts" it (marks the service running without spawning a duplicate that would die with EADDRINUSE). This adopt path calls setToolStatus(tool, "running") with no pid argument at all, so this.pid stays at its constructor default of null for the rest of that supervisor's life -- only the spawn path (a genuinely new child process) ever sets a real pid. In production this "adopt" path is common, not an edge case: any embedded sidecar service (cliproxy, 9router, bifrost, mux) whose child process survives a `systemctl --user restart omniroute.service` gets adopted by the new supervisor instance on the next start(), and its pid is lost from that point on -- even though the service is genuinely healthy and running. The observed symptom: a service shows state "running" but pid null, and something downstream that keys liveness tracking off pid eventually treats it as untrustworthy/stale despite nothing actually being wrong. Fix: resolve the real pid of the process holding the port (via `lsof -ti :<port>`, best-effort -- a lookup failure leaves pid null rather than blocking adoption) and record it the same way the spawn path does. An existing test asserted pid === null on adopt. That was accurate for the old behavior but reflected a missing resolution, not a deliberate "adopted services never get a pid" design choice -- updated its assertion to match the corrected behavior and added a dedicated regression test proving the resolved pid matches the real process actually bound to the port. |
||
|
Markus Hartung
|
cc17b304ab |
fix(sse): Gemini TPM/RPD quota classification + combo cooldown-wait resilience (#8213)
* fix(sse): Gemini TPM classification, combo-cooldown-wait for auto/quota-share, and target-timeout floor
Gemini TPM/RPM 429s were misclassified as QUOTA_EXHAUSTED because
sanitizeErrorMessage() truncates to the first line, hiding Google's
metric name and retry hint on lines 2-3. Added a rawMessage field
(internal-only, never reaches the client) and classifyGeminiQuotaMetricFromText()
to classify from the untruncated text, reordered ahead of the generic
credits/daily-quota checks.
Widened comboCooldownWaitEnabled (wait out a short transient cooldown
instead of crystallizing a 429/503) from quota-share-only to also cover
auto-strategy combos, and raised the wait ceiling to 65s/130s-budget/90s-cap
to match Gemini's ~60s TPM/RPM windows.
The per-target timeout (DEFAULT_COMBO_TARGET_TIMEOUT_MS, 120s) was shorter
than the new 130s cooldown-wait budget, so a target could get cut off
mid-wait with a synthetic 524 instead of completing the retry. Added
resolveComboTargetTimeoutMsForCombo()/isComboCooldownWaitEligible() in
comboConfig.ts to raise the per-target floor to budgetMs+buffer only for
wait-eligible strategies (auto/quota-share), verified live: a 12-request
concurrent burst against a TPM-exhausted combo went from 2/12 succeeding
(10 x 524) to 12/12 succeeding with zero 503/524.
Also: liveGeminiShared.ts's sendAndValidate now fails fast on a 503
instead of retrying past it, and the health dashboard + request logger
surface TPM stats alongside RPM/RPD.
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
* fix(sse): combo-exhausted rejection logs now capture request body + attempted models
recordRejectedRequestUsage() (the fast path for combo requests that never
reach handleChatCore, e.g. all targets locked by resilience cooldown)
hardcoded provider: "-" and never passed a request body to saveCallLog(),
so /dashboard/logs entries for these failures were nearly useless for
debugging: no way to see the client's request or which models were tried.
- recordRejectedRequestUsage() now accepts requestBody and persists it
through the existing saveCallLog() artifact mechanism (same path
handleChatCore's own logging uses).
- Added summarizeComboAttemptedModels(), which reads the combo's own model
list (always available, unlike the response's combo-diagnostics headers —
a model-level resilience-lockout skip never touches the
exhaustedProviders/exhaustedConnections sets those headers are built
from) to populate a real "provider" value instead of "-".
- Wired both into the call site in src/sse/handlers/chat.ts.
NOTE: unrelated to the Gemini TPM/combo-cooldown-wait fix on this branch —
landed here per operator request, to be split into its own branch/PR.
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
* feat(sse): synthetic streaming keep-alive event + 5-minute Gemini cooldown-wait ceiling
Many clients enforce a first-SSE-byte timeout, which made it unsafe to wait
out a longer upstream rate-limit cooldown on a streaming request — the
client would abandon the connection before any bytes arrived. This landed
in two parts:
1. Synthetic startup "thinking" event (OpenAI chat/completions format):
the already-existing withEarlyStreamKeepalive wrapper (open-sse/utils/
earlyStreamKeepalive.ts, wired into /v1/chat/completions, /v1/messages,
/v1/responses since #2544) opens the SSE stream immediately once a
request runs past its threshold, but only ever sent empty/no-op
keepalive frames. Added a `startupFrame` option (defaults to
`keepaliveFrame` — zero behavior change unless a route opts in) so the
very first frame can carry real content instead. Wired
OPENAI_STARTUP_THINKING_FRAME (a reasoning_content delta: "OmniRoute:
got request, sending to provider") into /v1/chat/completions only —
Claude Messages and Responses API formats both require a preceding
envelope event (message_start / response.created) that a synthetic
pre-dispatch frame can't safely fabricate without risking a duplicate
envelope once the real stream arrives, so those two routes keep their
existing (safe, proven) keepalive frames unchanged.
2. Raised the "wait out a known cooldown, then retry" ceiling to 5 minutes
for both retry mechanisms, now that a client-side first-byte timeout is
no longer a risk on the (opted-in) route:
- comboCooldownWait (auto/quota-share combos, open-sse/services/combo.ts):
maxWaitMs hard clamp raised 90s -> 300s (src/lib/resilience/settings/
normalize.ts); defaults raised to maxWaitMs:90s/maxAttempts:5/
budgetMs:300s. comboConfig.ts's resolveComboTargetTimeoutMsForCombo
already derives the per-target timeout floor from budgetMs, so it
tracks the new ceiling with no further changes.
- waitForCooldown (direct, non-combo model requests, src/sse/handlers/
chat.ts): this mechanism had NO cumulative cap before — only a
per-wait cap (maxRetryWaitMs) and a retry count (maxRetries), so
maxRetries x maxRetryWaitMs could exceed 5 minutes with no ceiling.
Added a budgetMs field (mirrors comboCooldownWait) to
WaitForCooldownSettings/CooldownAwareRetrySettings, threaded a
requestRetryBudgetLeftMs tracker through chat.ts's requestAttemptLoop
(mirrors combo.ts's comboCooldownBudgetLeftMs), and made
getCooldownAwareRetryDecision refuse to wait once the cumulative
budget is exhausted even if the single wait is under maxRetryWaitMs.
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
* fix(sse): extend the synthetic keep-alive thinking event to /v1/responses
Live incident (OpenClaw, log id 1784407081908-cbc24f): a /v1/responses
request to gemini/gemma-4-31b-it took 56s to produce a first byte and the
client disconnected (499 request_signal_aborted) — the same client-first-byte-
timeout problem the previous commit fixed for /v1/chat/completions, but
/v1/responses only had the generic bare-comment keepalive (no content), so it
wasn't covered.
Added RESPONSES_STARTUP_THINKING_FRAME: a self-contained synthetic reasoning
item (response.output_item.added -> reasoning_summary_part.added ->
reasoning_summary_text.delta -> reasoning_summary_part.done), opened AND
closed within this one frame rather than left dangling — it never carries a
response_id, so it can't collide with the real upstream response's own
independent response.created lifecycle that follows. Mirrors the abbreviated
delta+part.done close pattern open-sse/utils/stream.ts's own
emitSyntheticResponsesReasoningSummary already uses for real mid-stream
reasoning content.
Wired into src/app/api/v1/responses/route.ts via the startupFrame option
added in the previous commit.
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
* fix(sse): combo cooldown-wait vars reset every setTry, crystallizing a bogus 503 instead of waiting
Live incident (log id 1784416706646-51): a request to the "default" combo
(strategy=auto, maxSetRetries=3) hit a real Gemini TPM 429 on both gemma-4
targets, correctly classified as a short 40s rate_limit lockout — then
crystallized a 503 "all upstream accounts are inactive" in 6.9s instead of
ever reaching the cooldown-aware wait.
Root cause: `lastError`/`earliestRetryAfter`/`lastStatus` were declared with
`let` INSIDE the `for (setTry...)` loop body, so they reset to null at the
start of every set-try. When both targets lock out on setTry 0, every
subsequent setTry (1..maxSetRetries) pre-skips both targets via the
isModelLocked check with no real dispatch — so on the FINAL setTry (the only
one whose values the post-loop decision reads, since it's gated behind
`if (setTry < maxSetRetries) continue`), lastStatus was null, hitting the
"!lastStatus" branch (ALL_ACCOUNTS_INACTIVE 503) and completely bypassing the
comboCooldownWaitEnabled / earliestRetryAfter wait logic — even though a
real 429 with a known ~40s retry-after WAS observed on setTry 0.
This bug predates today's Gemini TPM work (any combo with maxSetRetries > 0
whose targets all lock out on the first pass was affected) but was masked in
existing tests: the "auto strategy (2 models...)" regression test uses
maxSetRetries: 0, so it only ever runs ONE setTry iteration and never
exercises the reset-on-retry path. It also explains why the dedicated
12-concurrent-request burst test passed cleanly — with concurrent requests,
timing variance meant some request's FINAL setTry iteration still had a live
target to dispatch to, giving lastStatus/earliestRetryAfter fresh data. A
single isolated request has no such luck.
Fix: hoist lastError/earliestRetryAfter/lastStatus to just inside
dispatchWithCooldownRetry, before the setTry loop, so they persist across
set-tries (still reset fresh on each recursive dispatchWithCooldownRetry()
call after a wait, which is correct). recordedAttempts/fallbackCount/
exhaustedProviders etc. are intentionally left per-iteration (unrelated to
this bug).
New regression test in tests/unit/combo-quota-share-cooldown-wait.test.ts
reproduces the exact live scenario (2 targets, both lock out on setTry 0,
maxSetRetries: 3) — confirmed red (503) against the pre-fix code, green
(200, waits and retries) against the fix.
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
* test(sse): extend live Gemini workload to Responses API + add large-context TPM test
Two additions to the live Gemini test suite, both live-verified against the
dev instance:
1. sendAndValidate() (tests/integration/liveGeminiShared.ts) now accepts an
apiFormat: "chat" | "responses" parameter, building the Responses-API
request shape (input array, max_output_tokens) and parsing its SSE events
(response.output_text.delta / response.reasoning_summary_text.delta /
response.completed) via the new readResponsesSSEStream(). Wired into two
new tests in live-gemini-workload.test.ts ([30]/[31]), mirroring the
existing Chat Completions streaming coverage. Verified live: 24/25 + 5/5
payloads succeeded end-to-end through the new code path (the one failure
was a ~300s test-client fetch timeout unrelated to the Responses API code
itself — a separate, not-yet-addressed test-harness limitation).
2. genHugeContextMessage() builds a single message large enough (~4
chars/token estimate) to approach or exceed Gemini's free-tier TPM ceiling
(16000 input tokens/min for gemma-4) by itself. Every other prompt
generator in this file tops out around 1-2k tokens — nowhere near that
ceiling — so none of the existing workload tests ever exercised a REAL TPM
429, only RPM-style rate limiting. tests/integration/gemini-large-context-tpm.test.ts
sends two ~12-13k-token requests back-to-back (comfortably exceeding
16000/min together) to exercise the full path against production Gemini:
TPM classification, the comboCooldownWait retry, and the synthetic
keep-alive frame on a genuinely slow request.
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
* fix(sse): abandoned combo target dispatch now observes its own per-target timeout, fixing a permanent "pending" dashboard leak
Live incident (dashboard log id 1784418258231-14961a, reported as "an ongoing
request even though there's already a 200"): a combo target dispatch
abandoned by comboTargetTimeoutMs (open-sse/services/combo/targetTimeoutRunner.ts)
left a permanent phantom "pending" entry in the dashboard, even after the
overall combo request had already succeeded via a different retry.
Root cause: chatCore.ts's createStreamController — and everything downstream
that depends on it (withRateLimit's Promise.race against Bottleneck,
acquireAccountSemaphore) — only ever watches clientRawRequest.signal, which
is the ORIGINAL client's request signal (set once via buildClientRawRequest
and reused unchanged across every target dispatch in a combo). It has no
connection to targetTimeoutRunner.ts's OWN AbortController
(target.modelAbortSignal), which is what actually fires when
comboTargetTimeoutMs (300s) elapses. src/sse/handlers/chat.ts's
handleSingleModel bridge between combo.ts and handleSingleModelChat received
`target.modelAbortSignal` but silently dropped it — never forwarded it
anywhere. So when a target got abandoned (e.g. stuck inside a wedged
Bottleneck rate-limiter queue, see the WEDGED force-reset log line from the
same incident), its per-target timeout fired and let the COMBO move on and
retry successfully elsewhere — but the abandoned dispatch's own promise
chain never learned it had been superseded, so it hung forever waiting on a
signal that was never going to fire, and trackPendingRequest(false) (the
finalize call) never ran.
Fix: thread target.modelAbortSignal through as a new modelAbortSignal
runtimeOption, and merge it into clientRawRequest.signal (via the existing
mergeAbortSignals helper from open-sse/executors/base.ts) right before
dispatch, so an abandoned target's own promise chain now observes its abort
and can reach its cleanup path — new resolveDispatchClientRawRequest() makes
this mechanically testable in isolation.
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
* fix(sse): combo cooldown-wait state recording, rate-limit wedge recovery, OpenAI-format SSE error frames
Five related fixes surfaced by live incidents (dashboard log ids 1784457764961-73,
1784465227489-a2cbc0, 1784504040241-6f8b9a) while validating the Gemini TPM/cooldown-wait
work on this branch against real OpenClaw traffic:
- combo.ts: the model-lockout bail-out branches in dispatchWithCooldownRetry never
recorded lastStatus, so once every target in a set hit an existing lockout the final
check crystallized a bogus ALL_ACCOUNTS_INACTIVE 503 instead of reaching the
cooldown-wait decision, even with a real 429 + short retry-after observed.
- combo.ts/combo/types.ts: the "all credentials cooling down" pre-dispatch rejection
(buildModelCooldownBody) nests its retry hint as error.retry_after/reset_seconds, not
the top-level retryAfter every other 429 shape uses — combo's extraction only read the
latter, so earliestRetryAfter stayed null for this shape even after lastStatus was fixed.
- rateLimitManager.ts: the wedge-recovery watchdog used disconnect(), which releases the
heartbeat timer but never rejects jobs already QUEUED on that instance — orphaned
dispatches hung until the outer ~300s per-target timeout, well past real clients'
patience. Switched to stop({ dropWaitingJobs: true }), safe because the wedge condition
already requires RUNNING===0 && EXECUTING===0.
- earlyStreamKeepalive.ts: the in-band error frame emitted after committing to a 200 SSE
stream was hardcoded to Anthropic's `event: error` convention for every route, including
the OpenAI-format ones (/v1/chat/completions, /v1/responses) where that framing is
either invisible or malformed to a plain data-line parser. Added per-route
OPENAI_CHAT_ERROR_FRAME / OPENAI_RESPONSES_ERROR_FRAME and wired them in.
- chatCore.ts: persisted a synthetic clientResponse error body even when the client had
already disconnected (AbortError) before that body was ever computed — misleading the
dashboard into showing "what the client received" for a response that was never sent.
Also: RequestLoggerDetail.tsx — Provider/Client Event Stream panes lost their collapse
toggle when StreamSection replaced the collapsible PayloadSection (
|