Chat admission sampled host-wide /proc/pressure/memory, so a swapping
Docker host 503'd idle containers with resource_pressure. Prefer this
unit's cgroup memory.pressure and keep the host file as fallback.
* fix(db): never TRUNCATE-checkpoint a live WAL
A live TRUNCATE checkpoint rewrites the shared wal-index while other
processes hold it mapped; dereferencing the stale mapping SIGBUSes the
process. Two production crashes six hours apart, coredump stack in
better-sqlite3 native memcpy (issue #13973).
Remove the periodic TRUNCATE scheduler. Runtime checkpoints are
PASSIVE-only, which move pages without changing the wal-index geometry,
while TRUNCATE stays on the shutdown path where reclaiming the file is
safe. The 256MB size guard now warns instead of escalating to a live
TRUNCATE, busy PASSIVE ticks feed the persisted busy telemetry that the
TRUNCATE tick used to carry, and a positive
OMNIROUTE_WAL_TRUNCATE_INTERVAL_MS logs a one-time deprecation warning.
Signed-off-by: Minxi Hou <houminxi@gmail.com>
* fix(db): RESTART the WAL when it exceeds the size guard
The 256MB size guard only warned, so a live WAL could keep growing until
the next restart. wal_checkpoint(RESTART) starts a new WAL file without
rewriting the mapped wal-index, which is what SIGBUS'd the process when
we used TRUNCATE under traffic.
Related to #13973.
Signed-off-by: Minxi Hou <houminxi@gmail.com>
* docs: drop a fake TRUNCATE env name from the WAL guard row
Backticks around TRUNCATE made the env/docs checker treat it as a
variable. The VACUUM rows next to it were never part of this change
and are not in the base docs.
Signed-off-by: Minxi Hou <houminxi@gmail.com>
---------
Signed-off-by: Minxi Hou <houminxi@gmail.com>
Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@gmail.com>
* fix(ci): document OMNIROUTE_STRIP_SYSTEM_PREAMBLE (env/docs contract base-red)
* fix(ci): allowlist COMBO_LOOP_SAFETY_TIMEOUT_MS as a doc-only source constant
The env/docs contract gate had a SECOND violation on the release tip, added after
this branch was cut: #13857's comboTimeoutMs narrative in ENVIRONMENT.md cites
COMBO_LOOP_SAFETY_TIMEOUT_MS, which is a source constant
(open-sse/services/combo/comboPredicates.ts:35 — `10 * 60 * 1000`), not an
operator-facing env var. The doc regex captured the SHOUTY_NAME and reported it
as documented-but-missing-from-.env.example.
DOC_ONLY_ALLOWLIST already exists for exactly this class (see CLI_COMPAT_OMITTED_PROVIDER_IDS,
LOCAL_ONLY_API_PREFIXES, VACUUM). Gate now reports all three directions in sync.
* feat(dashboard): expose comboTimeoutMs next to Target timeout
The runtime already applied config.comboTimeoutMs as the whole-combo
wall-clock budget (0 = 10-minute hang-stop). Schema treated it as an
unknown passthrough key and the dashboard only painted Target timeout,
so operators could not raise the 15-step failover ceiling from the UI.
Declare comboTimeoutMs on comboRuntimeConfigSchema, mount both knobs in
the combo editor Advanced panel and Combo defaults, and keep
comboTimeoutMs longer than targetTimeoutMs so failover still has time.
Signed-off-by: Minxi Hou <houminxi@gmail.com>
* docs(changelog): attach #13857 to comboTimeoutMs fragment
Signed-off-by: Minxi Hou <houminxi@gmail.com>
* test(dashboard): name comboTimeoutMs store as milliseconds
The input is seconds; the stored config field is milliseconds. The old
title said "in seconds" while asserting 1_200_000.
Signed-off-by: Minxi Hou <houminxi@gmail.com>
* chore(i18n,changelog): translate #13857 keys into all locales and drop the CHANGELOG hunk
---------
Signed-off-by: Minxi Hou <houminxi@gmail.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* feat(compression): make Lite tool-result truncation length configurable
Lite truncated tool results at a hardcoded 2000 characters. Coding-agent
payloads (file reads, crash dumps) lost the middle of the content with no
supported way to raise the cap.
Honor lite.maxToolLength from settings, then OMNIROUTE_LITE_MAX_TOOL_LENGTH,
then 2000. Existing installs keep the old length.
Related to #13178.
Signed-off-by: Minxi Hou <houminxi@gmail.com>
#13178 stays open.
* compression/lite: keep a stored cap when a step or toggle write is incomplete
An out-of-range step maxToolLength was still a number, so it hid a valid
global cap and fell through to env. A toggle-only settings PUT replaced
the whole lite row and dropped the stored cap. Save treated an out-of-range
number like a cleared field. Reject the bad Save, merge omitted caps, and
use null to clear.
Related to #13178.
Signed-off-by: Minxi Hou <houminxi@gmail.com>
* compression/lite: stop dashboard copy from hard-coding a 2000-char cap
The page overlays schema descriptions from i18n. Updating only
LITE_SCHEMA left operators seeing "over 2,000 characters" after the
cap became configurable. Also assert the Save error string, not the
Save button.
Related to #13178.
Signed-off-by: Minxi Hou <houminxi@gmail.com>
* chore(changelog): move #13915 entry to a changelog.d fragment
---------
Signed-off-by: Minxi Hou <houminxi@gmail.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(db): reclaim freed pages incrementally instead of a blocking VACUUM in the cleanup scheduler (#12821)
startCleanupScheduler() ran a synchronous whole-database VACUUM on the event
loop whenever a cleanup pass deleted at least one row - 30 s after every
start and every 6 h. With node:sqlite that blocks every route (/healthz
included) for the duration: 7 min 55 s on a 540 MB storage.sqlite to reclaim
six rows. It also bypassed vacuumScheduler, the app-level owner of full
VACUUMs and the operator's scheduledVacuum / vacuumHour settings.
cleanup.ts no longer issues a full VACUUM. After each pass reclaimFreedPages()
branches on PRAGMA auto_vacuum:
- INCREMENTAL: drain the freelist with PRAGMA incremental_vacuum(N) in ~1 MiB
batches (N from page_size), pausing between batches for as long as the last
one took (<=250 ms), PASSIVE checkpoint every 64 batches and a TRUNCATE
checkpoint at the end so the main file shrinks in WAL mode; hard caps of
2048 batches / 30 s per pass, the remainder waits for the next pass.
- FULL: nothing to do, SQLite reclaims on commit.
- NONE: incremental_vacuum is a no-op, so record a request via the new
vacuumScheduler.requestFullVacuum(); the rebuild runs in the configured
window (or via the Storage page button). scheduledVacuum=never is honored.
vacuumScheduler persists fullVacuumRequestedAt / fullVacuumRequestReason,
clears them on the next successful runNow(), and hydrates from key_value
before an early request so it cannot clobber a persisted lastRunAt.
Loop robustness: db.exec() rather than pragma() (bun:sqlite's all() steps a
zero-column pragma once), SQLITE_BUSY/LOCKED and a handle closed under the
pass stop it quietly, other errors stop it with partial progress logged.
Also drops the duplicate cleanupProxyLogs() call in the scheduled pass -
runAutoCleanup() already covers proxy_logs.
Tests: new tests/unit/db/cleanup-reclaim-freed-pages.test.ts (INCREMENTAL
drain/pause/checkpoint, page_size-derived batch, caps, FULL no-op, NONE
defers and leaves page_count untouched, runScheduledCleanupPass() path);
vacuum-scheduler.test.ts covers requestFullVacuum persistence, restart
survival and clearing; cleanup-column-fix.test.mjs now asserts
incremental_vacuum and the absence of a full VACUUM statement.
* chore(changelog): name the #12821 fragment after its PR (#12830)
* fix(db): extract reclaimFreedPages into its own module and fix full-suite regressions
Split the #12821 incremental-vacuum reclamation logic out of cleanup.ts
into src/lib/db/reclaimFreedPages.ts (re-exported for callers/tests) so
cleanup.ts stays under the file-size cap after the #13011 reconciliation
merge grew it past the 1200-line threshold.
Also fixes two full-suite failures surfaced by running the
cleanup/vacuumScheduler/db-health suite post-merge (not just this PR's
own 3 test files, per the plan-file's mandatory item):
- tests/unit/cleanup-column-fix.test.mjs scanned cleanup.ts's raw source
for the PRAGMA incremental_vacuum invariant, which now lives in the
extracted module — updated to scan both files.
- tests/unit/db/cleanup-reclaim-freed-pages.test.ts asserted the freelist
count is byte-for-byte unchanged when auto_vacuum=NONE. The tip's
runAutoCleanup() now also runs cleanupCompressionRunTelemetry(), which
lazily creates its table on first use (ensureCompressionRunTelemetryTable)
— a legitimate one-time page cost from a freshly migrated DB, unrelated
to reclaimFreedPages()'s own behavior. Loosened the assertion to a small
tolerance while keeping the page_count assertion that actually guards
against a full rebuild.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* chore(db): drop the reclaimable-bytes VACUUM gate test superseded by incremental reclaim
tests/unit/vacuum-reclaimable-threshold.test.ts pinned cleanup.ts's
vacuumAfterCleanup()/getReclaimableBytes()/getVacuumMinReclaimableBytes()
(#13079). This branch removes the inline post-cleanup full VACUUM entirely in
favour of reclaimFreedPages() (#12821), which reads the same freelist_count /
page_size signal and defers a full VACUUM to the vacuum scheduler when
auto_vacuum=NONE. With those three exports gone the file cannot compile, and
the behaviour it guarded no longer exists.
---------
Co-authored-by: insoln <is@careerum.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(sse): Anthropic OAuth 403 "Request not allowed" is a per-request refusal, not a ban
A single upstream 403 on the `claude` OAuth connection was classified
FORBIDDEN and written as the terminal `banned` connection state
(chatCore -> writeTerminalStatus). From then on every request to that
provider was short-circuited with "All 1 connection(s) banned by
upstream - please reconnect in the dashboard" without touching Anthropic,
until an operator reconnected.
Anthropic's OAuth surface answers a small fraction of otherwise-valid
requests with 403 {"type":"permission_error","message":"Request not
allowed"}. On the reporting install the same token returned 200 forty
seconds before the 403 and again right after the connection was
re-enabled; a revoked or expired token is a 401 authentication_error, not
this. It is a refusal of one request, not of the credential.
Classify it as the new non-terminal PROVIDER_ERROR_TYPES.REQUEST_REJECTED
(scoped to provider `claude` and the "Request not allowed" body) and list
that type in authTerminalStatus.isNonTerminalProviderError, mirroring the
Cloudflare FINGERPRINT_REJECTION precedent. The combo layer still falls
through to the next target for the failing request; the connection stays
active for the next one. Any other claude 403 keeps its previous
classification.
Tests: error-classifier.test.ts covers the Anthropic body, the
gateway-flattened "[403]: Request not allowed" message, the same body from
a non-Anthropic provider (still FORBIDDEN), other claude 403s (unchanged),
and the helpers; anthropic-request-not-allowed-not-a-ban.test.ts pins
resolveTerminalConnectionStatus() -> null for the new type even with a
`permanent` fallback verdict, and `banned` for a generic claude 403.
* fix(sse): cooldown with backoff and streak escalation for REQUEST_REJECTED (#12859)
Not "ignore the 403" either: if Anthropic ever made "Request not allowed"
systematic, re-sending every request into it would be the wrong thing to
do to an OAuth account. chatCore now handles REQUEST_REJECTED explicitly:
- exclude the connection via setConnectionRateLimitUntil for a growing
cooldown (5 -> 15 -> 45 min) so a sporadic refusal costs minutes, not a
reconnect, and a systematic one cannot become a stream of 403s;
- escalate to the terminal `banned` state only for 3 refusals within a
60-minute window (services/requestRejectedStreak.ts, in-memory per
connection; a restart forgets the streak, erring towards more cooldowns
rather than an operator-undone ban), with a last_error that says so;
- probe-origin failures record but never cool down or ban (#9817).
The existing "request not allowed" text rule (5 s) is unaffected:
markAccountUnavailable skips a connection that already has a future
rateLimitedUntil, so the minute-scale cooldown written here wins.
Tests: request-rejected-streak.test.ts pins the window/threshold/backoff
arithmetic; anthropic-request-not-allowed-cooldown-escalation.test.ts drives
the real chat route against a mocked 403 upstream on a `claude` OAuth
connection: 300 s cooldown, then 900 s, then banned on the third refusal;
a different claude 403 body still bans on the first response.
* chore(changelog): name the #12859 fragment after its PR (#12864)
* refactor(sse): move the REQUEST_REJECTED branch into a chatCore leaf; register its tests for mutation coverage
chatCore.ts is frozen at 5984 lines by the file-size ratchet; the branch
body now lives in open-sse/handlers/chatCore/requestRejectedFailure.ts
(chatCore: 5974 -> 5983). stryker.conf.json tap.testFiles gains the two new
DB-backed tests so their mutant kills count (check:mutation-test-coverage).
* fix(sse): count refusal episodes, reset on success, keep the dashboard honest (#12859 review)
Review findings on the first cut of the REQUEST_REJECTED handling:
- A burst of in-flight requests that all got the 403 within seconds
produced streak 1, 2, 3 and a ban from one upstream event. The streak
now counts cooldown *episodes*: a refusal that lands while the
connection is already excluded is the same event and is not counted.
- Nothing reset the streak on a healthy response, so sporadic refusals
on a busy install could still accumulate to a ban. chatHelpers'
onRequestSuccess now clears it (only a real success does - the recovery
tick's clearAccountError is an elapsed cooldown, not a success).
Clearing the cooldown by hand in the dashboard clears it too.
- The third rung of the ladder was unreachable (the third refusal
escalates): the ladder is now 5 -> 15 min, sourced from COOLDOWN_MS next
to the existing 5 s "request not allowed" rule, with a note on why that
rule is superseded for claude. The 60-min window becomes a 24 h
staleness bound - "consecutive" is defined by successes, not by time.
- Probe-origin refusals no longer touch the streak (#9817).
- The cooldown is written like every other connection-level cooldown:
ISO rateLimitedUntil + testStatus "unavailable" (+ lastErrorAt), so the
dashboard shows the countdown and the recovery tick restores "active".
- One refusal is re-seeded from the persisted row after a restart so a
crash loop cannot reset the count on every boot.
Docs: RESILIENCE_GUIDE terminal states + CODEBASE_DOCUMENTATION resilience
row mention the streak module. Tests cover the burst, the success reset,
the seed, and the ISO/unavailable shape end-to-end through the chat route.
* chore(sse): drop unrelated Prettier churn in auth.ts / providers route
* style(api): keep providers route Prettier-clean
* refactor(sse): share the "exclude connection for a cooldown" leaf between GEO_BLOCKED, GCP_PROJECT_REQUIRED and the new branch
The release tip moved chatCore.ts to its frozen 5984 lines, so the
REQUEST_REJECTED branch cannot add a single net line. The GEO_BLOCKED and
GCP_PROJECT_REQUIRED branches were the same eight statements with different
constants and log wording; both now call
open-sse/handlers/chatCore/connectionCooldown.ts::excludeConnectionForCooldown
(behaviour, probe guard and log lines preserved verbatim). chatCore.ts ends
9 lines below the base it branched from.
* chore(chatCore): tighten the cooldown comments to keep the file under its size ceiling
After merging release/v3.8.51, chatCore.ts sat at 6150 lines against a
frozen ceiling of 6146. Condense the explanatory comments this PR added
to the GEO_BLOCKED and GCP_PROJECT_REQUIRED branches; no code change.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: insoln <is@careerum.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(combo): answer 503 + Retry-After, not 404, when a weighted pool is only cooling down
The weighted strategy filters targets before dispatch (open circuit breaker,
provider cooldown, model lockout, availability probe) and drops them
silently. When that emptied the pool the host returned the 404
"Combo has no executable targets" with the "switch combo / reconnect the
missing providers" recovery hint — for a pool that was configured,
connected and merely cooling down. Claude Code renders a 404 from
/v1/messages as "this model may not exist".
- targetResolution.ts: the eligibility predicate now reports which gate
excluded a target and, for the resilience gates, the remaining time;
the exclusions of fully-excluded steps travel out of
resolveWeightedSelection. When the weighted pool ends empty and at least
one exclusion is a resilience timer, the pipeline returns an early 503
and logs the reasons at warn level.
- pinRecovery.ts: buildAllTargetsCoolingDownResponse() — 503
`all_targets_cooling_down`, Retry-After = earliest exclusion to lapse,
every excluded target in diagnostics.excluded, `wait` recovery hint with
retry_after_seconds; formatPreDispatchExclusions() for the log line.
- error.ts: `all_targets_cooling_down` joins the public error identifiers.
- A pool emptied only by the availability probe keeps the 404.
- docs: RESILIENCE_GUIDE debugging entry; changelog fragment.
* chore(changelog): name the fragment after PR #12956 and link issue #12954
* refactor(combo): keep weighted exhaustion below complexity ratchet
---------
Co-authored-by: insoln <is@careerum.com>
* fix(resilience): lock the exact model, not the quota family, on 5xx model-lockout failures
A 5xx model-lockout failure — a transport error (terminated, EHOSTUNREACH,
connect timeout), an upstream server error, or OmniRoute's own synthesized
502 from quality validation — is evidence about one model endpoint at that
moment, not about the account's quota family. recordModelLockoutFailure()
wrote it under the quota-family key regardless, so for codex (whose family
key is the whole `codex` scope, i.e. every gpt-5* model) one empty stream
on gpt-5.6-luna removed gpt-5.6-sol and gpt-5.6-terra from routing too,
for 2–30 min with exponential escalation, while the quota was untouched.
- exactModelLock.ts: resolveLockoutScope(status, explicit) — 429/403/402
(and 404, already narrowed by getModelLockKey) keep the family key; any
other status uses the exact provider/connection/model key. An explicit
`scope` option still wins.
- recordModelLockoutFailure() resolves the scope once for key + lock fn.
- decayModelFailureCount() now walks every key shape (family, not_found,
exact) so success-decay reaches exact-scope locks; null model stays a
no-op.
- getAllModelLockouts() parses the `exact:` marker out of the key so the
Model Cooldowns card lists the bare model and can clear it by that name.
- docs: RESILIENCE_GUIDE §3 key-scope-by-status; changelog fragment.
* chore(changelog): name the fragment after PR #12957 and link issue #12955
---------
Co-authored-by: insoln <is@careerum.com>
* security(deps): pin and verify tls-client native artifacts
* docs(changelog): link tls-client provenance PR
* security(runtime): harden TLS and public error boundaries
* security(runtime): resolve CodeQL error-boundary findings
* security(lmarena): close public stream error boundary
* fix(lmarena): normalize public error statuses
* chore(quality): rebaseline chatCore.ts for the surviving log-boundary hardening
open-sse/handlers/chatCore.ts 6219 -> 6287. This is the one part of #11742 that
survived the rebase: sanitizeErrorMessage on the plugin onError hook, on the
semaphore-timeout path and on failureMessage before it reaches console.log and
the call log, sanitizeUpstreamDetails on the malformed-response log, and
getSafeErrorMetadata + try/catch where hostile (Proxy) metadata could throw.
That is the LOG boundary, which is broader than Hard Rule #12 (responses). The
rest of the PR was dropped as already landed on the tip.
* test(catalog): pin the 2026-09-02 free-tier re-audit facts for gemini, ollama-cloud, groq, nara and mistral
* feat(providers): add xKiro (5M tokens/day free plan, 39 pinned free models)
* fix(catalog): re-audit gemini, ollama-cloud, groq, nara and mistral against official pages
* docs(providers): xKiro in the provider reference, counts and free-tier headline (~1.66B)
* fix(catalog): restore the console-verified Mistral 1B pool and harden its regression test
* docs(free-tiers): move headline to the re-audited ~1.50B and refresh pool counts
* chore(free-tiers): retire stale Groq free-tier text and preset model; fix catalog header
* docs(providers): align the remaining visible provider/executor counts with the catalog
* docs(providers): align remaining free-tier count chips and metadata
* docs(free-tiers): state the evidence-comment rule honestly and retire the last "14.4K RPD" Groq texts
* docs(free-tiers): retire the stale Gemini onboarding quota text
* docs(free-tier): refresh catalog-entry counts to 442 after base sync
* docs(providers): re-sync provider and free-tier counts after merging release/v3.8.51
* docs(providers): re-sync residual counts after the base merge
* docs(free-tiers): restore README spacing lost in the merge and re-sync the guide counts
* docs(free-tiers): re-sync numbers after merging release/v3.8.51 (Cerebras reclassified upstream)
* fix(docs): keep the NaraRouter plans endpoint out of the API-path checker; rebaseline gateways.ts (+3)
* chore(quality): rebaseline gateways.ts file-size cap for the xKiro entry (+20)
---------
Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
* fix(providers): remove the chipotle/pepper provider (#13131)
amelia.chipotle.com (the reverse-engineered Amelia chat-widget backend
chipotle/pepper-1 talked to) now returns 404 on every route, including
root, from its Azure Application Gateway — confirmed live 2026-09-15.
This regressed from a WS handshake timeout (#4037, June 2026) to a
fully decommissioned host, so the upstream protocol cannot be fixed.
Owner decided to retire the provider entirely (Option B), following
the phind/kluster quiet-removal precedent: no REMOVED_PROVIDERS.md
entry (reserved for operator takedowns), just a one-line note under
FREE_TIERS.md "Removed / no free tier".
Removed every surface: executor, registry entry, executors/index.ts
and providers/index.ts wiring, noauth provider catalog entry,
ProviderIcon generic-fallback set, the autoCombo exclusion-list
comment, the chipotle_error code from the sanitizer allowlist,
PROVIDER_REFERENCE.md (regenerated), and every doc/test reference.
Regression test: tests/unit/issue-13131-chipotle-provider-removed.test.ts
asserts the provider is fully gone from the executor registry, the
provider REGISTRY and the noauth catalog, and that the executor module
no longer resolves — not a live-network repro (flaky/third-party).
Several existing tests used "chipotle" only as a generic noAuth-provider
example (proxy scoping, error classification, onboarding, fallback
text) with no chipotle-specific behavior under test; those were
re-pointed at another still-existing noAuth provider
(cloudflare-playground / duckduckgo-web) rather than weakened.
* test(providers): document the agnes-cn/chipotle count coincidence (#13131)
provider-node-reserved-prefix.test.ts's REGISTRY id+alias walk was
already red on the base tip (414 vs. expected 412) from agnes-cn
(#13399, +id/+alias). Removing chipotle's REGISTRY id/alias in this
PR nets it back to 412, making the test pass again without a numeric
edit — record why in a comment so it doesn't read as an untracked
coincidence later.
On 2026-09-16 eight feature PRs added 61 keys to src/i18n/messages/en.json and
stamped `__MISSING__:<en>` into all 65 locales instead of translating.
check-new-key-coverage accepted the marker as "the key reached the locale", so
nothing blocked the PRs, and the blocking real-translation ratio gate then failed
on the release tip for everybody (pt-BR 3.2 % > 2.5 % + 0.5).
- scripts/i18n/check-new-key-coverage.mjs: a leaf whose value starts with
`__MISSING__:` is judged exactly like an absent leaf; the FAIL message names
the marker as the cause and prints the per-locale sync-ui-keys command and the
parallel runner. Header/JSDoc updated.
- tests/unit/i18n-new-key-coverage.test.ts: "a new key that only carries a
__MISSING__ marker is flagged" (was the inverse case, which encoded the old
contract); the other six cases unchanged and green.
- scripts/i18n/translate-new-keys.sh (+ `npm run i18n:translate-new-keys`):
committed, detached-safe runner — flock queue, N workers (default 5), 3
attempts per locale of `sync-ui-keys.mjs --translate-markers --batch-size=40`,
per-locale logs/.exit + batch.log/batch.status/batch.rc/batch.pid under
_artifacts/i18n-new-keys/, non-zero exit while any locale still carries a
marker, refuses to start (exit 2, names the five OMNIROUTE_TRANSLATION_* vars)
when the backend env is absent. Reads only the OMNIROUTE_TRANSLATION_* lines
of the repo .env; kills nothing, matches nothing by name.
- docs: QUALITY_GATES.md (gate table + check-new-key-coverage section) and
I18N.md (gate table + "Translating the keys a branch adds" subsection).
The implementation/port/merge skills reference the new shared snippet
`.agents/skills/_shared/i18n-translate-new-keys.md` (skills repo, separate).
Second `npm run release:reconcile` pass on `release/v3.8.50..release/v3.8.51`
(091589089c..c0f92ec98a, 916 non-merge commits, 877 merged PRs):
- fold the 173 changelog.d fragments accumulated since #12971 under
`## [3.8.51]` and delete them
- generate bullets for the 58 cycle commits that had no fragment
(4 features / 42 fixes / 12 maintenance), each with the merged PR link and
`— thanks @author`
- link 137 fragment bullets to the PR of the commit that added them and
credit the author; two prefix/origin mismatches reviewed (#12945→#13392,
#13001→#13379, both maintainer rebaselines of other people's PRs)
- refresh "Release by the numbers" + Top-25 and regenerate the
`### 🙌 Contributors` hall (112 external contributors + maintainer; every
non-bot author of the 877 merged PRs present)
- closed-PR credit audit for the window: nothing to add (#13215→#13361 and
#13059→#13690 are still open, #12998 was independently fixed earlier by
#12853); no human co-author trailers, no commits without a PR
- resync the 58 i18n CHANGELOG mirrors
Gates: check:changelog-integrity OK, check:docs-sync PASS.
Both src/lib/machineToken.ts::getActiveSalt() and its mirror in
bin/cli/utils/cliToken.mjs derived the CLI/management bearer token as
HMAC-SHA256(raw machine-id, salt) with a checked-in literal default salt
("omniroute-cli-auth-v1"). Since /etc/machine-id is commonly world-readable,
any local user who never set OMNIROUTE_CLI_SALT could derive the same
bearer token as the server.
getActiveSalt() now generates a random 64-char-hex salt on first use and
persists it under <DATA_DIR>/cli-token-salt.json (falling back to the
literal only when neither the env override nor a persisted/writable salt
can be established). Both implementations use the same resolution order
and the same wx-flag create-race handling so the CLI and server keep
deriving the same token. OMNIROUTE_CLI_SALT stays the explicit operator
override, unchanged.
Regression test: tests/unit/machine-token-random-salt-13679.test.ts
* fix(sse): bound active streams without terminal events
* fix(sse): derive the active-stream ceiling from the largest registered model budget
The watchdog is a hard lifetime cap that never resets on bytes, so a flat
15-minute default killed models the registry already allows to run for 20
minutes (the Codex entries declare timeoutMs: 1_200_000). The default is now
that maximum plus a one-minute margin, and a new test re-derives the maximum
from the registry so a future larger budget fails the gate instead of silently
re-opening the bug.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(quality): clear the release/v3.8.51 base-reds
19 failing unit tests plus the API Route Typecheck and mutation-test-coverage
gates, all reproduced on the clean tip before touching anything.
Ten of the failures share one cause. #13452/#13798 made `*-compatible-*`
buildUrl() refuse a connection with no baseUrl instead of quietly defaulting to
the real OpenAI/Anthropic API — which would ship the operator's stored key to a
public third party. The guard is right; three fixtures still built those
connections unhydrated, and one of them put baseUrl at the top level of
credentials, where the chat path never reads it.
The rest:
- modelDiscovery.ts missed the VertexModelMetadataProvenance cast that its
read-path twin in db/models/synced.ts already had — both written by #12471.
- A provider-test regexp carried raw 0x00/0x1f bytes, which makes git, GitHub
and ripgrep treat the file as binary. Same character class, written
with escapes instead of the bytes themselves.
- #13399 (Agnes AI China) adds "agnes-cn" + "agnescn": the only two provider
prefixes since the count was last set (412 -> 414). Everything else added in
that range is model ids.
- The free-tier budget card SVG was stale (443 -> 452 models); regenerated by
its own script.
- Three new tests were missing from stryker.conf.json tap.testFiles, so the
mutants they kill did not count.
Three guards asserted syntax rather than the invariant they protect, and broke
when the source legitimately changed. Each was re-expressed and then verified by
mutating the source back:
- #2331 required modelEffort to head the rawEffort chain; #13556 deliberately
put the server-selected force rule first. The real invariant is relative —
modelEffort outranks the defaults a client injects — and it still trips when
explicitReasoning is moved ahead of it.
- The OAuth loopback guard matched the isLocalhost arm literally; #9944 added
`&& !opts?.manualLoopback`. It now matches the arm whatever guards it, and
still fails when the hint stops being built.
- The i18n scanner flagged dynamically-built keys — t("effort." + mode) reaches
it as a literal prefix, never a string. It now accepts a prefix that resolves
to a namespace holding messages, and still fails when the namespace is gone.
tests/unit/sse-auth.test.ts (#12080) expected a bare null where #13879 now
returns the key-policy diagnostic — the same sentinel shape the terminal-state
path has used since #12441. The assertion was rewritten to the constraint #12080
actually protects: nothing usable comes back and neither connection leaks. The
contract risk that remains — those sentinels are truthy, and executeWebSearch
treats any truthy value as a credential — is filed as #13945 rather than
widened into this PR.
Refs #13866
* fix(quality): clear the second wave of release/v3.8.51 base-reds
The tip moved 13 commits while the first pass was running and brought its own
reds. All reproduced locally on the merged tree first.
vitest 4.1.11 -> 5.0.0 in the #13661 development-group bump is a major, and
vitest 5 moved `vite` from a dependency to a peerDependency. This repo only ever
declared `vite` under `overrides`, which pins a version but installs nothing, so
`npm ci` stopped providing it and the Vitest job died at startup with
ERR_MODULE_NOT_FOUND. Declared as the devDependency it actually is — the same
^8.0.16 the override already pinned, and what @vitejs/plugin-react asks for as a
peer — and regenerated the lockfile: 684 lines added, none changed.
#12909 filtered a mapped array with `toolCall is JsonRecord`, but the element
type is the tool-call literal or null, and a predicate's type has to be
assignable to the parameter's (TS2677). Narrowed by the element's own type
instead; the literal still satisfies JsonRecord at the return.
#12906 added `|| result.errorCode === "empty_response"` to the stream-failure
condition and Prettier rewrapped it, so the #8928 probe — which located the
branch by an exact four-line string — stopped finding it. It now matches on what
the branch tests rather than how it is typeset, and still fails when the
eviction call is removed.
probe-7293 is the visible half of a real conflict, filed as #13948. #7293 merges
a mid-array system into index 0; #12908, landed later, demotes it to "user" in
place instead. Both target the same constraint and only one can win, and the
combination also reorders: the pre-translation hoist moves the turn forward
expecting it to stay a system message, then the demotion converts it where it
now sits, ahead of the conversation. Choosing between the two strategies is a
product call, not a base-red one, so the test was realigned to assert the half
that protects the caller — the instruction survives, as a user turn — and pins
the current ordering with a pointer to the issue, so the eventual decision shows
up as a deliberate test change instead of a silent regression.
Refs #13866, #13948
* fix(quality): allowlist vite, rebaseline tip growth, drop a dead import
Third pass on the release/v3.8.51 base-reds. Declaring `vite` in the previous
commit was correct but incomplete: check-deps is a human review point against
typosquatting, so a newly declared package has to be vouched for by name.
Recorded in dependency-allowlist.json with why it is needed — the official Vite
build tool, already pinned through overrides, and a required peer of both
vitest 5 and @vitejs/plugin-react. That also turns check-deps.test.ts green.
check-file-size went red on nine files. One is mine: sse-auth.test.ts grew when
the #12080 assertion was rewritten. Three of the four assertions I had added
were redundant with the strict deepEqual that follows them, so they are gone and
the file grows by 4 lines instead of 8; the cap absorbs the rest.
The other eight are production and test files this PR does not touch, grown by
other work and never rebaselined — which is the whole reason a base-red drain
exists. Each is attributed to the commit that grew it: #12906 (chat.ts,
chatHelpers.ts, proxyFetch.ts, stream.ts), #12904 + #12910 (chatCore.ts), and
batch_api.test.ts from the same wave. Two of them predate the wave entirely and
were already over cap on 3d5baf13 — imageGeneration.ts (#13748) and
roundRobinCombo.ts (#13776) — so they were base-reds hiding behind a gate that
only surfaced them once the tip was merged in. Both are recorded separately from
the wave so the history stays honest about when each cap actually moved.
Note for whoever reads the gate next: it counts one line more than `wc -l`,
since it measures split length rather than newlines.
Finally, #13290 replaced rmSync with cleanupTempDataDir in
zcode-executor.test.ts but left the import behind, which the frozen-warning
ESLint gate rejects. Removed.
Refs #13866
1,104 mirrors rewritten over five passes of run-translation on the 22-source core set: the 14 sources edited since their translation, the 322 mirrors that were still English copies, and the frontmatter the old extractor leaked into the newer locales' bodies. The pipeline now caches per-`## `-section hashes and retranslates only changed sections, never reuses a section that is still English, rebuilds English-copy / leaked mirrors even when the source is unchanged, merges the state on save (parallel runs), and the drift gate (scoped to the core set) is blocking. Final audit: 0 stale, 0 English copies, 0 leaked frontmatter across 1,430 core mirrors.
⚠️ base-red inherited: #12732
* fix(sse): inject global system prompt post-translation for codex/Responses path
codex/Responses requests carry input[]+instructions, not messages[]. The
existing injectSystemPrompt runs PRE-translation (chatCore.ts) and only
handles messages[]/system fields, so the Global System Prompt (After Prompt =
suffixPrompt) never reached the provider for codex — verified 0/84 call logs
while the catalog base_instructions reached 84/84.
Add injectSystemPromptPostTranslation() and call it after prepareUpstreamBody
on the resolved messages[]. With multiple system/developer messages (codex
normalises its per-item developer roles to system), prefix goes on the FIRST
and suffix on the LAST so the After Prompt retains the highest recency
position — the semantics injectSystemPrompt's single-findIndex buries.
Also wire OMNIROUTE_SYSTEM_INSTRUCTION_APPEND on the /v1/messages (Claude
Messages -> OpenAI Chat Completions) translation path. The directive was
previously only wired on the Responses API path, so DeepSeek-V4 kept leaking
English planning/chain-of-thought into the content field on Claude Code
sessions that route through /v1/messages. Mirror the openai-responses.ts
pattern: append to string system, append a text block for array content, or
unshift a new system message when none exists.
Tests: 23/23 (19 system-prompt incl. 6 postTranslation + codex regression;
4 claude-to-openai directive append). typecheck:core clean.
* test(sse): reproduce global prompt double injection — single-injection contract tests
* fix(sse): unify global prompt injection to single post-translation pass
* fix(sse): carry single global-prompt injection across claude/gemini/responses target shapes
* fix(sse): restore global-prompt coverage for carrier-less targets via gated pre-translation pass
* fix(sse): cover codex/gemini source shapes in the carrier-less pre-translation gate
* fix(types): preserve generic system prompt return
* fix(sse): correct file reference in claude-to-openai.ts comment and add changelog fragment
Points the #reasoning-bilingual comment at the real companion file
(translator/response/openai-to-claude.ts's directivePreambleStripper.ts
from #12905) instead of the nonexistent "openai-responses.ts", and adds
the changelog fragment referenced in the PR body but missing from the
diff.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: Jihyun Son <jihyun.son@sk.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
The warmup scheduler's circuit-breaker keys were written to Redis without `REDIS_KEY_PREFIX`, so they escaped OmniRoute's namespace and could collide with another app sharing the instance — the one Redis surface the prefix wasn't reaching. Probe: 2/2 pass in `tests/unit/lib/warmupScheduler/redisCircuitBreakerStorePrefix.test.ts`, covering both the prefixed case and the unset/blank case where keys must stay unchanged.
**Batch validation** — boarded with the other 10 PRs of your batch into one worktree cut from `release/v3.8.51`; every PR verified as an ancestor of the combined HEAD before validating.
- Focused tests across all 11 PRs: **104/104 pass** on the combined tree.
- Gates on the combined tree: `check-changelog-integrity` PASS, `check-complexity` PASS, `check-cognitive-complexity` PASS, `typecheck:core` PASS, `check:open-sse-typecheck` PASS.
- `check-file-size` is red, but reproduces with byte-identical line counts on the pure `release/v3.8.51` tip (`open-sse/handlers/imageGeneration.ts` 3304, `open-sse/services/combo/roundRobinCombo.ts` 1221, `open-sse/utils/stream.ts` 3115). Inherited base-red, nothing added by this batch — it is also why this PR's "Fast Quality Gates" check was red.
**Reconciled** — this PR was `CONFLICTING`. The conflict was in `docs/reference/ENVIRONMENT.md` and purely additive: the release tip had inserted `APP_BIND_HOST` / `QDRANT_BIND_HOST` / `BIFROST_BIND_HOST` rows directly above the `REDIS_KEY_PREFIX` row you edited. Kept both sides — the tip's three new rows and your updated description naming the warmup circuit breaker — then merged the current release branch in (120a92f6) and re-ran your focused test on the reconciled tree: 2/2 pass. No line of your diff was dropped.
Thanks, @datrixlab — you also updated `.env.example`, `docs/ops/REDIS_PRODUCTION_CONFIG.md` and `ENVIRONMENT.md` alongside the code, which is why the only thing left to do here was a mechanical conflict resolution.
Lands the combined-board reconciliation of today's JxnLexn wave as one follow-up: i18n fill for #12471/#13555's new keys (real vi translations), free-tier count 446→452, file-size rebaseline for #13556. check-new-key-coverage PASS; only the three pre-existing file-size reds remain.
Merged after a maintainer rework that kept every one of @hartmark's commits intact.
**What the rework added:** the auto-clean of terminal batch checkpoints and expired file content is gated behind a default-off feature flag (`BATCH_AND_FILE_AUTO_CLEANUP_ENABLED`, `defaultValue: "false"`, documented in `docs/reference/FEATURE_FLAGS.md` and described in all 66 locales) so the release default keeps today's behaviour and operators opt in; the DB handle leak in the test was fixed so the Node runner exits cleanly.
Validated as a combined board first (this PR merged with the 21 siblings of the same wave on the release tip): eslint with the frozen suppressions, typecheck:core, check:open-sse-typecheck, complexity, cognitive-complexity, changelog-integrity, i18n new-key coverage, docs-counts, docs-sync, migration-numbering, provider-consistency and a duplicate-identifier audit all green, plus 176 passing / 0 failing focused node:test cases across the 25 test files the wave touches and the dashboard test under Vitest (2/0). Then re-validated alone on the fresh tip before this merge: conflicts re-resolved, file sizes rebaselined for this PR's own growth, eslint and this PR's focused tests re-run.
Thank you — the cleanup itself is exactly the kind of maintenance that stops a data dir from growing forever.
Merged after a maintainer rework that kept every one of @patrykkopycinski's commits intact — including the two refactors you pushed later (extracting the adaptive-effort wiring out of `chatCore.ts` and reading `x-omniroute-effort` inside the wiring module), which were merged into the rework rather than overwritten.
**What the rework added:** the adaptive-effort wiring is scoped to OpenAI-dispatch requests only (the claim in `docs/routing` was corrected to match), and `defaultReasoningEffort` was widened to accept `auto` explicitly instead of relying on a loose string.
Validated as a combined board first (this PR merged with the 21 siblings of the same wave on the release tip): eslint with the frozen suppressions, typecheck:core, check:open-sse-typecheck, complexity, cognitive-complexity, changelog-integrity, i18n new-key coverage, docs-counts, docs-sync, migration-numbering, provider-consistency and a duplicate-identifier audit all green, plus 176 passing / 0 failing focused node:test cases across the 25 test files the wave touches and the dashboard test under Vitest (2/0). Then re-validated alone on the fresh tip before this merge: conflicts re-resolved, file sizes rebaselined for this PR's own growth, eslint and this PR's focused tests re-run.
Thank you — gateway-resolved, per-turn pinned effort is a real feature, and the header contract makes it usable from every harness.
Merged after a maintainer rework that kept every one of @hartmark's commits intact.
**What the rework added:** the reclaimable-space gate for the auto-cleanup VACUUM sits behind a default-off feature flag so the release default is unchanged, with the flag documented in `docs/reference/FEATURE_FLAGS.md` and described in all 66 locales; the rest is your change as submitted.
Validated as a combined board first (this PR merged with the 21 siblings of the same wave on the release tip): eslint with the frozen suppressions, typecheck:core, check:open-sse-typecheck, complexity, cognitive-complexity, changelog-integrity, i18n new-key coverage, docs-counts, docs-sync, migration-numbering, provider-consistency and a duplicate-identifier audit all green, plus 176 passing / 0 failing focused node:test cases across the 25 test files the wave touches and the dashboard test under Vitest (2/0). Then re-validated alone on the fresh tip before this merge: conflicts re-resolved, file sizes rebaselined for this PR's own growth, eslint and this PR's focused tests re-run.
Thank you — gating VACUUM on reclaimable pages instead of row count is the right signal.
Merged. Settling on `kv_after_text` while a trailing `exec_mcp` call is still pending drops the tool call entirely — the client then sees a finished turn that never ran the tool. Correct place to fix it.
Validated as a combined board first (this PR merged with the 21 siblings of the same wave on the release tip): eslint with the frozen suppressions, typecheck:core, check:open-sse-typecheck, complexity, cognitive-complexity, changelog-integrity, i18n new-key coverage, docs-counts, docs-sync, migration-numbering, provider-consistency and a duplicate-identifier audit all green, plus 176 passing / 0 failing focused node:test cases across the 25 test files the wave touches and the dashboard test under Vitest (2/0). Then re-validated alone on the fresh tip before this merge: conflicts re-resolved, file sizes rebaselined for this PR's own growth, eslint and this PR's focused tests re-run.
Thank you.
Merged. Opt-in is what makes this safe to ship: thinking models that need a floor get one, everyone else sees no change in behaviour, and the env var is documented in `.env.example` and `ENVIRONMENT.md` rather than being folklore.
Validated as a combined board first (this PR merged with the 21 siblings of the same wave on the release tip): eslint with the frozen suppressions, typecheck:core, check:open-sse-typecheck, complexity, cognitive-complexity, changelog-integrity, i18n new-key coverage, docs-counts, docs-sync, migration-numbering, provider-consistency and a duplicate-identifier audit all green, plus 176 passing / 0 failing focused node:test cases across the 25 test files the wave touches and the dashboard test under Vitest (2/0). Then re-validated alone on the fresh tip before this merge: conflicts re-resolved, file sizes rebaselined for this PR's own growth, eslint and this PR's focused tests re-run.
Thank you.
Merged after a maintainer rework that kept every one of @HouMinXi's commits intact.
**What the rework added on top of the contribution:** the new DB health-check behaviour is gated behind a default-off feature flag (`src/shared/constants/featureFlagDefinitions.ts`, `defaultValue: "false"`), documented in `docs/reference/FEATURE_FLAGS.md` with the description key carried into all 66 locales, so the release default is unchanged and the new bounds only apply when an operator opts in. The optional-FTS5 migration set was reconciled by hand with the "180" entry that landed meanwhile (`src/lib/db/migrationRunner/constants.ts`).
**Carried from your rebased head:** the `/api/db/health` local-only classification in `src/server/authz/routeGuard.ts` plus its `routeGuard` assertion — `runManagedDbHealthCheck()` forks native diagnostics into a child process, so Hard Rules #15/#17 apply. Re-verified here: 37 pass / 0 fail.
Validated as a combined board first (this PR merged with the 21 siblings of the same wave on the release tip): eslint with the frozen suppressions, typecheck:core, check:open-sse-typecheck, complexity, cognitive-complexity, changelog-integrity, i18n new-key coverage, docs-counts, docs-sync, migration-numbering, provider-consistency and a duplicate-identifier audit all green, plus 176 passing / 0 failing focused node:test cases across the 25 test files the wave touches and the dashboard test under Vitest (2/0). Then re-validated alone on the fresh tip before this merge: conflicts re-resolved, file sizes rebaselined for this PR's own growth, eslint and this PR's focused tests re-run.
Thank you for the depth of this one — the resource-bounds suite and the sql.js startup/backup coverage are the kind of tests that keep a database layer honest.
`check:env-doc-sync` is failing on the release tip, which fails "Docs Gates
(fast-path)" on every open PR against release/v3.8.51 (base-red #13866).
Both gaps come from #13679:
- `OMNIROUTE_CLOUD_SYNC_ENFORCE_SIGNATURE` is read in src/lib/cloudSync.ts but
was in neither .env.example nor ENVIRONMENT.md. Documented with the behaviour
the code actually implements: opt-in rejection of an UNSIGNED response when no
local secret is configured, default off for v3.8.x back-compat, and a present
signature always verified — and always rejected when
OMNIROUTE_CLOUD_SYNC_SECRET is unset — regardless of the flag.
- `CDP_PROXY_TOKEN` was in .env.example but missing from ENVIRONMENT.md. Added to
the ChatGPT Web (Codex) table next to CHATGPT_WEB_CODEX_CDP_URL, in that
section's language, describing the X-Omni-Cdp-Token header the sidecar expects
and the compose-network isolation that applies when it is unset.
Docs only, no code change.
Verified on this branch: check:env-doc-sync reports all three directions in sync
(817 vars in .env.example, 834 in ENVIRONMENT.md); check:docs-sync passes;
check:docs-counts reports only pre-existing soft drift.
Merged in the 2026-09-16 sweep of the maintainer's own open PRs, at the owner's explicit instruction. No push was made to the PR branch: the merge took the head as the owning session left it (verified OPEN, non-draft and MERGEABLE against the release tip immediately before merging).
New blocking gate `i18n:check-keys` (`scripts/i18n/check-key-completeness.mjs`): every locale catalog must carry exactly the key set of `en.json`, whatever the age of the key. The percentage and new-key gates let batch 1 (#13044) ship 43 keys short and batch 2 (#13660) 10 keys short. Wired into the i18n-ui-coverage job; documented in QUALITY_GATES.md and the i18n guide (post-merge re-sync, retranslation with pinned names).
⚠️ base-red inherited: #12732
Restores `memory_id` to the `memory_fts_au` trigger's `WHEN` clause and adds an FTS5 rebuild to the memory cleanup pass.
This repairs a regression that landed yesterday: migration 178 guarded the trigger with `old.content IS DISTINCT FROM new.content OR old.key IS DISTINCT FROM new.key`, which drops the `memory_id` term the insert path depends on. `createMemory` inserts the row, the AFTER INSERT trigger stores an auto-assigned FTS5 rowid, and the follow-up `UPDATE memories SET memory_id = rowid` is what re-syncs FTS — an update that touches neither `content` nor `key`. With 178 alone that update stopped firing, so newly created memories drifted out of the FTS index and keyword/hybrid search silently returned nothing for them. Migration 180 adds the third term back.
Maintainer note before merge: the cleanup half now runs `rebuild` on every pass rather than `optimize` only when rows were deleted — accepted as-is; it is bounded by the memory table size and the trigger fix removes the bloat source that motivated it.
Validated as a combined board first (this PR merged with the 11 siblings of the same batch on the release tip): eslint on every changed file with the suppressions file, typecheck:core, check:open-sse-typecheck, complexity, cognitive-complexity, changelog-integrity, i18n new-key coverage, docs-sync, migration-numbering, provider-consistency and a duplicate-identifier audit all green, plus 275 passing / 0 failing focused node:test cases across the 28 test files the batch touches. Then re-validated alone on the fresh tip before this merge: conflicts re-resolved, file sizes rebaselined for this PR's own growth, eslint and this PR's focused tests re-run.
Thanks @HouMinXi!
Groq is now excluded from Chrome TLS impersonation in both the direct and the proxied dispatch branch, even when `TLS_FINGERPRINT_PROVIDERS` is unset or explicitly lists it — Cloudflare answers the spoofed fingerprint with 1010 `browser_signature_banned` (#13225).
Scope is contained: the whole path is behind `ENABLE_TLS_FINGERPRINT`, which defaults to off, so nothing changes for operators who never opted in.
Validated as a combined board first (this PR merged with the 11 siblings of the same batch on the release tip): eslint on every changed file with the suppressions file, typecheck:core, check:open-sse-typecheck, complexity, cognitive-complexity, changelog-integrity, i18n new-key coverage, docs-sync, migration-numbering, provider-consistency and a duplicate-identifier audit all green, plus 275 passing / 0 failing focused node:test cases across the 28 test files the batch touches. Then re-validated alone on the fresh tip before this merge: conflicts re-resolved, file sizes rebaselined for this PR's own growth, eslint and this PR's focused tests re-run.
Thanks @HouMinXi!
The opencode executor classifies rate-limited 429 bodies (`classify429`, with real tests) and, when a whole account wave is exhausted, returns the last real upstream 429 — status, body, `Retry-After` and quota headers intact — so the provider error rules (monthly-quota cooldown) keep working.
Maintainer rework before merge (kept the idea, no default behavior change):
- The original stopped the cross-account wave at the first classified 429 and replaced the response with a synthetic one that dropped the body and headers; stopping early is now opt-in behind `OPENCODE_RATE_LIMITED_429_EARLY_STOP` (default off), the rate-limited account is still cooled down, the body is read as a bounded 8 KiB prefix from a clone and the original is never consumed, and the unused `status` input is gone.
Validated first on the combined board of all 38 PRs of this batch (10 merged as-is, 28 after the maintainer rework) on top of release/v3.8.51 c0f92ec: typecheck:core, check:open-sse-typecheck and check:dashboard-typecheck clean; ESLint clean on every changed file; file-size (rebaselined for the combined growth), complexity, cognitive-complexity, changelog-integrity, docs-counts, docs-sync, migration-numbering and i18n new-key gates green; 735 focused node:test cases with the only batch-caused failure (a flag-count assertion) fixed. Then re-validated alone on the fresh release tip right before this merge: ESLint on the changed files, typecheck:core, check:open-sse-typecheck, the file-size/complexity/changelog gates and this PR's own tests.
Thanks @maxmad64bis!
Behind the new `MISTRAL_AMBIGUOUS_401_SOFT_LOCKOUT` flag (default off), a bare Mistral 401 (`{"detail":"Unauthorized"}`, identical for a revoked key and an exhausted quota) gets a retryable cooldown instead of parking the connection as `expired`; after three soft strikes within an hour the next bare 401 parks it, so revocation still converges.
Maintainer rework before merge (kept the idea, no default behavior change):
- The predicate is shared with the connection-test module instead of duplicated; the squeezed 139-char line that dodged the file-size gate is formatted normally and the growth is rebaselined honestly with an annotation.
Validated first on the combined board of all 38 PRs of this batch (10 merged as-is, 28 after the maintainer rework) on top of release/v3.8.51 c0f92ec: typecheck:core, check:open-sse-typecheck and check:dashboard-typecheck clean; ESLint clean on every changed file; file-size (rebaselined for the combined growth), complexity, cognitive-complexity, changelog-integrity, docs-counts, docs-sync, migration-numbering and i18n new-key gates green; 735 focused node:test cases with the only batch-caused failure (a flag-count assertion) fixed. Then re-validated alone on the fresh release tip right before this merge: ESLint on the changed files, typecheck:core, check:open-sse-typecheck, the file-size/complexity/changelog gates and this PR's own tests.
Thanks @maxmad64bis!
Behind the new `OPENCODE_TRANSIENT_FAILOVER_BACKOFF` flag (default off), after two consecutive transient upstream failures the opencode rotation pauses before each later account (1.5s, 3s, 6s, capped at 10s per request) instead of hammering the upstream.
Maintainer rework before merge (kept the idea, no default behavior change):
- The pause honors the client abort signal (no dispatch after a disconnect), the failed attempt's body is cancelled before sleeping, `transientRetryDelayMs` now uses its arguments, and the sleep is injectable so the tests run without real timers.
Validated first on the combined board of all 38 PRs of this batch (10 merged as-is, 28 after the maintainer rework) on top of release/v3.8.51 c0f92ec: typecheck:core, check:open-sse-typecheck and check:dashboard-typecheck clean; ESLint clean on every changed file; file-size (rebaselined for the combined growth), complexity, cognitive-complexity, changelog-integrity, docs-counts, docs-sync, migration-numbering and i18n new-key gates green; 735 focused node:test cases with the only batch-caused failure (a flag-count assertion) fixed. Then re-validated alone on the fresh release tip right before this merge: ESLint on the changed files, typecheck:core, check:open-sse-typecheck, the file-size/complexity/changelog gates and this PR's own tests.
Thanks @maxmad64bis!
Behind the new `OPENCODE_USER_BLOCKED_ROTATION` flag (default off), a 403 or 451 carrying `user_blocked` on a proxied opencode account rotates at most once to the next account instead of being returned as-is.
Maintainer rework before merge (kept the idea, no default behavior change):
- 403 and 451 are handled by one predicate (the original returned 451 without rotation), the refused account gets a cooldown and joins the tried-set, and the response body of the attempt rotated away from is cancelled.
Validated first on the combined board of all 38 PRs of this batch (10 merged as-is, 28 after the maintainer rework) on top of release/v3.8.51 c0f92ec: typecheck:core, check:open-sse-typecheck and check:dashboard-typecheck clean; ESLint clean on every changed file; file-size (rebaselined for the combined growth), complexity, cognitive-complexity, changelog-integrity, docs-counts, docs-sync, migration-numbering and i18n new-key gates green; 735 focused node:test cases with the only batch-caused failure (a flag-count assertion) fixed. Then re-validated alone on the fresh release tip right before this merge: ESLint on the changed files, typecheck:core, check:open-sse-typecheck, the file-size/complexity/changelog gates and this PR's own tests.
Thanks @maxmad64bis!
Behind the new `OPENCODE_RESPONSES_STALL_ROTATION` flag (default off): a streamed Responses reply with no first body byte within `RESPONSES_FIRST_BYTE_TIMEOUT_MS` (15s) cools the account and rotates once; a second stall fails fast instead of waiting the 80s readiness timeout.
Maintainer rework before merge (kept the idea, no default behavior change):
- The TLS first-byte watchdog from #12656 is restored byte for byte (the PR had changed its pump, timer and cancel); the stall guard lives in its own module.
- Proxy-less multi-account setups now rotate the same way as proxied ones (the original threw for them), a client abort during the wait rethrows instead of rotating, and the env var is documented as flag-only.
Validated first on the combined board of all 38 PRs of this batch (10 merged as-is, 28 after the maintainer rework) on top of release/v3.8.51 c0f92ec: typecheck:core, check:open-sse-typecheck and check:dashboard-typecheck clean; ESLint clean on every changed file; file-size (rebaselined for the combined growth), complexity, cognitive-complexity, changelog-integrity, docs-counts, docs-sync, migration-numbering and i18n new-key gates green; 735 focused node:test cases with the only batch-caused failure (a flag-count assertion) fixed. Then re-validated alone on the fresh release tip right before this merge: ESLint on the changed files, typecheck:core, check:open-sse-typecheck, the file-size/complexity/changelog gates and this PR's own tests.
Thanks @maxmad64bis!
Behind the new `PROXY_HEALTH_BLOCKED_RESETS_STREAK` flag (default off), a probe the target refuses (401/403/429) resets the proxy's consecutive-failure streak, so a proxy that clearly relays is not marked dead by spaced-out real failures.
Maintainer rework before merge (kept the idea, no default behavior change):
- The original reversed the deliberate #10654 policy for everyone; with the flag off a refusal stays neutral, and the existing assertions are restored. The stale JSDoc and the wrong "any relayed response resets" comment are fixed (5xx stays inconclusive).
- The source-grep test became a real sweep test: a local relay answering 403 drives fail → blocked → fail with auto-disable, in both flag modes.
Validated first on the combined board of all 38 PRs of this batch (10 merged as-is, 28 after the maintainer rework) on top of release/v3.8.51 c0f92ec: typecheck:core, check:open-sse-typecheck and check:dashboard-typecheck clean; ESLint clean on every changed file; file-size (rebaselined for the combined growth), complexity, cognitive-complexity, changelog-integrity, docs-counts, docs-sync, migration-numbering and i18n new-key gates green; 735 focused node:test cases with the only batch-caused failure (a flag-count assertion) fixed. Then re-validated alone on the fresh release tip right before this merge: ESLint on the changed files, typecheck:core, check:open-sse-typecheck, the file-size/complexity/changelog gates and this PR's own tests.
Thanks @maxmad64bis!
Behind the new `PROXY_POOL_EGRESS_OBSERVATION` flag (default off): a line under each proxy pool showing how many distinct egress IPs actually served it over 24h, backed by `GET /api/settings/proxies/pool/egress-observation`.
Maintainer rework before merge (kept the idea, no default behavior change):
- The route validates its query with Zod (unknown `scope` → 400 instead of silently `global`), error bodies go through `errorResponse()`, the OpenAPI entry documents security, parameters and responses, and the three UI strings exist in every locale.
Validated first on the combined board of all 38 PRs of this batch (10 merged as-is, 28 after the maintainer rework) on top of release/v3.8.51 c0f92ec: typecheck:core, check:open-sse-typecheck and check:dashboard-typecheck clean; ESLint clean on every changed file; file-size (rebaselined for the combined growth), complexity, cognitive-complexity, changelog-integrity, docs-counts, docs-sync, migration-numbering and i18n new-key gates green; 735 focused node:test cases with the only batch-caused failure (a flag-count assertion) fixed. Then re-validated alone on the fresh release tip right before this merge: ESLint on the changed files, typecheck:core, check:open-sse-typecheck, the file-size/complexity/changelog gates and this PR's own tests.
Thanks @maxmad64bis!
`proxy_logs` records `upstream_status`, the HTTP status the provider actually returned through the proxy, instead of only success/timeout/error.
Maintainer rework before merge (kept the idea, no default behavior change):
- The migration collided with the tip (177 was already taken): renumbered to `179_proxy_logs_upstream_status.sql`, the runner's already-applied check moved to `case "179"` (the old `"177"` would have skipped the tip's own 177), migration count bumped to 176 in README, AGENTS.md, llm.txt and its mirrors (operator-approved).
- A new test runs the real migration runner on the real SQL files and fails with the old case number.
Validated first on the combined board of all 38 PRs of this batch (10 merged as-is, 28 after the maintainer rework) on top of release/v3.8.51 c0f92ec: typecheck:core, check:open-sse-typecheck and check:dashboard-typecheck clean; ESLint clean on every changed file; file-size (rebaselined for the combined growth), complexity, cognitive-complexity, changelog-integrity, docs-counts, docs-sync, migration-numbering and i18n new-key gates green; 735 focused node:test cases with the only batch-caused failure (a flag-count assertion) fixed. Then re-validated alone on the fresh release tip right before this merge: ESLint on the changed files, typecheck:core, check:open-sse-typecheck, the file-size/complexity/changelog gates and this PR's own tests.
Thanks @maxmad64bis!
Behind the new `PROXY_SKIP_RECENTLY_FAILED` flag (default off), pool rotation and the opencode account rotation remember a proxy that just failed (refused probe or 429) and skip it for a doubling cooldown instead of re-serving it immediately.
Maintainer rework before merge (kept the idea, no default behavior change):
- The original was on by default and re-queried the DB on every request while a member was set aside; selection now caches a refusal sequence number and re-runs the cascade once per set-aside event.
- `src/lib/db` no longer imports the heavy dispatcher for key normalization (a parity test guarantees the same key as `proxyConfigToUrl()`); `.env.example` and `ENVIRONMENT.md` document the default as false.
Validated first on the combined board of all 38 PRs of this batch (10 merged as-is, 28 after the maintainer rework) on top of release/v3.8.51 c0f92ec: typecheck:core, check:open-sse-typecheck and check:dashboard-typecheck clean; ESLint clean on every changed file; file-size (rebaselined for the combined growth), complexity, cognitive-complexity, changelog-integrity, docs-counts, docs-sync, migration-numbering and i18n new-key gates green; 735 focused node:test cases with the only batch-caused failure (a flag-count assertion) fixed. Then re-validated alone on the fresh release tip right before this merge: ESLint on the changed files, typecheck:core, check:open-sse-typecheck, the file-size/complexity/changelog gates and this PR's own tests.
Thanks @maxmad64bis!
GHSA-7pq4-8pvv-rx7r (critical). Every link of the reported chain held on the
release tip:
1. First boot without JWT_SECRET generates one and writes it in cleartext to
$DATA_DIR/server.env.
2. With no password configured, isAuthRequired() returned false for
POST /api/settings/require-login unconditionally — before the loopback
check — so any network peer could switch requireLogin off.
3. With requireLogin off, POST /api/settings/obsidian/webdav accepted an
arbitrary vault root and echoed freshly minted Basic credentials.
4. The WebDAV file service is served by the custom Node layer before Next.js,
outside the authz pipeline.
5. Pointing it at DATA_DIR reads server.env, and JWT_SECRET forges an
`{"authenticated":true}` admin session.
A second, worse problem surfaced while verifying: isLoopbackRequest() decided
"loopback" from nextUrl.hostname / the Host header, which the client controls.
`Host: localhost` from a remote address made the whole fresh-install bootstrap
reachable, not just the write path.
Three cuts, plus the root cause:
- isLoopbackRequest() now reads the trusted peer: the token-stamped real TCP
peer the custom server writes (peerStamp), then the pipeline's own locality
verdict once a stamp token exists, then a real socket peer. The bootstrap
write path honours the same constraint instead of returning false, and
managementPolicy hands down the peerContext verdict explicitly, because at
policy time the original request still carries client-supplied headers.
- Host is consulted only when the process has no stamp token at all — no
stamping server in front, which in practice means route handlers invoked
directly by the unit-test harness. Every supported runtime (run-next dev and
start, standalone-server-ws for Docker, the npm CLI and Electron) calls
ensurePeerStampToken() at boot, so there a signal-less request fails closed.
Without this fallback ~340 route tests that call handlers with
`new Request("http://localhost/…")` turned into 401s.
- /api/settings/obsidian joins ALWAYS_PROTECTED_API_PATHS: issuing and rotating
reusable WebDAV credentials is credential export, the same rationale as the
GHSA-62vw entry for the password reveal.
- enableObsidianVaultSync() refuses a vault that is, sits inside, or contains
DATA_DIR, comparing realpath-resolved paths so a symlink cannot dodge it.
Tests are red-first: remote stamped peer → auth required on the bootstrap
write; Host: localhost plus a forged locality header from a non-loopback
stamped peer → 401 through the full pipeline; the local operator keeps the
first-password flow; obsidian inventory and DATA_DIR overlap cases.
Behind `STREAM_EARLY_EOF_SIBLING_FAILOVER_ENABLED` (default off): after the bounded same-connection retry is spent, a stream that closed early fails over exactly once to a sibling connection.
Maintainer rework before merge (kept the idea, no default behavior change):
- The PR's own failover test was red on its head: the `/v1/chat/completions` route's early-stream keepalive dropped the `X-OmniRoute-Selected-Connection-Id` header on the first cold request. Tests now drive `handleChat()` directly; the assertion was kept.
- "One hop" was one hop per connection (a 3-connection pool made 4 dispatches); it is now a single sibling hop per request, and when the pool runs out the original `STREAM_EARLY_EOF` 502 is returned instead of a generic `bad_gateway`, so combo-level detection keeps working. The source-regex timeout test became a behavioral one; flag description in all 59 locales.
Validated first on the combined board of all 38 PRs of this batch (10 merged as-is, 28 after the maintainer rework) on top of release/v3.8.51 c0f92ec: typecheck:core, check:open-sse-typecheck and check:dashboard-typecheck clean; ESLint clean on every changed file; file-size (rebaselined for the combined growth), complexity, cognitive-complexity, changelog-integrity, docs-counts, docs-sync, migration-numbering and i18n new-key gates green; 735 focused node:test cases with the only batch-caused failure (a flag-count assertion) fixed. Then re-validated alone on the fresh release tip right before this merge: ESLint on the changed files, typecheck:core, check:open-sse-typecheck, the file-size/complexity/changelog gates and this PR's own tests.
Thanks @maxmad64bis!
Behind `STREAM_RECOVERY_TOOLCALL_ORDER_FIX` (default off), mid-stream continuation becomes tool-call safe: any tool call seen in the stream — in flight or finished — blocks a continuation, and an empty continuation stops after one attempt.
Maintainer rework before merge (kept the idea, no default behavior change):
- The empty-continuation short-circuit also ran with the flag off; it is now gated, so the flag-off path uses the whole budget exactly as before (regression test added).
- The latch re-arm that let a continuation fire after a completed `finish_reason: tool_calls` is gone; index-less tool calls on multi-choice payloads are now blocked too; ~150 lines of dead trace plumbing removed.
Validated first on the combined board of all 38 PRs of this batch (10 merged as-is, 28 after the maintainer rework) on top of release/v3.8.51 c0f92ec: typecheck:core, check:open-sse-typecheck and check:dashboard-typecheck clean; ESLint clean on every changed file; file-size (rebaselined for the combined growth), complexity, cognitive-complexity, changelog-integrity, docs-counts, docs-sync, migration-numbering and i18n new-key gates green; 735 focused node:test cases with the only batch-caused failure (a flag-count assertion) fixed. Then re-validated alone on the fresh release tip right before this merge: ESLint on the changed files, typecheck:core, check:open-sse-typecheck, the file-size/complexity/changelog gates and this PR's own tests.
Thanks @maxmad64bis!
Behind the new `PROTECTED_PRIORITY_INFRA_502_ENABLED` flag (default off), protected-priority combo stops caused by provably non-quota infrastructure (provider circuit open, predictive-TTFT latency) surface as 502 instead of a quota-looking 503.
Maintainer rework before merge (kept the idea, no default behavior change):
- The original branch made 502 the default for every stop, including model lockouts and cooldowns, and removed the #8133/#1731 provider-wide skip for 401/5xx without a connection id; both are restored with their regression tests untouched.
- Nineteen cases cover eight gate causes plus predictive latency, flag off and on.
Validated first on the combined board of all 38 PRs of this batch (10 merged as-is, 28 after the maintainer rework) on top of release/v3.8.51 c0f92ec: typecheck:core, check:open-sse-typecheck and check:dashboard-typecheck clean; ESLint clean on every changed file; file-size (rebaselined for the combined growth), complexity, cognitive-complexity, changelog-integrity, docs-counts, docs-sync, migration-numbering and i18n new-key gates green; 735 focused node:test cases with the only batch-caused failure (a flag-count assertion) fixed. Then re-validated alone on the fresh release tip right before this merge: ESLint on the changed files, typecheck:core, check:open-sse-typecheck, the file-size/complexity/changelog gates and this PR's own tests.
Thanks @maxmad64bis!
