* fix(monitoring): canonicalize provider aliases in health matrix
* fix(monitoring): canonicalize aliases in health autopilot
---------
Co-authored-by: tkgo11 <7.1800574e+07+tkgo11@users.noreply.github.com>
* fix(sse): close the synthetic keepalive reasoning item's output_item
RESPONSES_STARTUP_THINKING_FRAME (the /v1/responses early-keepalive
placeholder for slow-starting reasoning models) opened a synthetic
"rs_keepalive" reasoning item at output_index 0 and closed its nested
summary part (response.reasoning_summary_part.done), but never sent
response.output_item.done to close the item itself. The comment
claimed it was "closed within this one frame" — that was true for the
part, not the item.
Since this placeholder has no real upstream counterpart (the real
response starts an independent response.created lifecycle later and
never touches it), nothing else ever closes it. A client tracking open
items by output_index (as the Responses API spec requires — this is
exactly what OpenClaw's parser does) sees index 0 still open when the
real response's own output_item.added later reuses that same index,
and throws a collision.
Live incident (2026-08-13, reliably reproducing by 2026-08-14): traced
via a live tcpdump capture on the OmniRoute-dev container's network
namespace, correlated against the OpenClaw gateway journal and 10
separate real request/response pairs (all wire-clean on the response
side, ruling out provider corruption). The failing request's own
outbound payload confirmed a replayed reasoning item without
encrypted_content feeding a continuation call; the response wire bytes
for that exact exchange showed rs_keepalive's output_item.added at
index 0, then response.created/response.in_progress arriving *after*
it, then a second output_item.added reusing index 0 for the real
reasoning item — never preceded by an output_item.done for
rs_keepalive. Reported upstream as OpenClaw issue #123342 before the
OmniRoute-side root cause was found.
Fix: emit response.output_item.done for the synthetic item, matching
its already-buffered summary text, right after the summary part closes
and before the frame ends.
Test plan:
- tests/unit/early-stream-keepalive.test.ts: updated the frame-shape
test to assert the full 5-event closed sequence (added the missing
output_item.done and its field assertions); confirmed it fails
against pre-fix code (only 4 events) and passes after
- node --test tests/unit/early-stream-keepalive.test.ts,
tests/unit/earlyStreamKeepalive.test.ts,
tests/unit/keepalive-cleanup-8140.test.ts,
tests/unit/chat-body-admission.test.ts: 58 passed, 2 pre-existing
skips unrelated to this change (Node test runner
ReadableStream-error-simulation limitation)
- tsgo --noEmit: clean on both touched files
* fix(sse): allocate the keepalive output_index from a stack, not a literal
Follow-up to 03f8345ac. That commit patched the specific symptom (added
the missing response.output_item.done). This commit fixes the class:
RESPONSES_STARTUP_THINKING_FRAME hardcoded output_index: 0 as a literal
across five hand-written events, which is exactly how the missing-close
bug happened in the first place — nothing enforced that every open got
a matching close, so it silently didn't for months.
ResponsesOutputIndexStack (open-sse/utils/responsesOutputIndexStack.ts)
makes that structural: open() allocates the next sequential index,
close() must name the index being closed and throws if it doesn't match
the stack's top, and assertAllClosed() throws if anything is still open.
The keepalive frame now calls assertAllClosed() at module load, so a
future regression of this exact shape fails at import/boot time instead
of shipping a malformed stream to production and surfacing days later
as a live incident.
Also adds tests/helpers/assertResponsesOutputIndexLifecycle.ts: a
reusable version of the same invariant for replaying a full SSE event
sequence (not just checking one frame's own shape), mirroring what a
real client's output-index tracker enforces. Existing coverage for this
bug class (responses-reasoning-close-before-message-466.test.ts) only
asserted it by hand for one specific emitter (the real translator); nothing
generic existed for a hand-rolled synthetic frame like this keepalive to
be checked against, which is why its own test could pass while the actual
downstream contract still failed. Wired into
early-stream-keepalive.test.ts, including a test that concatenates the
keepalive frame with a plausible real subsequent response and asserts no
collision — the scenario that actually reproduced live, not just the
frame's own internal shape.
Test plan:
- tests/unit/responses-output-index-stack.test.ts (new): open/close/
assertAllClosed behavior, including the exact mismatch and
never-closed shapes this incident hit
- tests/unit/early-stream-keepalive.test.ts: existing frame-shape test
plus new collision-simulation test, both passing
- node --test across responses-output-index-stack, early-stream-keepalive,
earlyStreamKeepalive, keepalive-cleanup-8140, chat-body-admission:
65 passed, 2 pre-existing skips unrelated to this change
- tsgo --noEmit: clean on all touched files
---------
Co-authored-by: adevwithpurpose <adevwithpurpose@users.noreply.github.com>
* fix(docker): prefix cache mount ids with Railway service scope
Railway's Dockerfile builder rejects --mount=type=cache ids that lack
the s/<service-id>- prefix (dockerfile invalid, caught at syntax
validation before any build step runs). Prefix all 7 cache mount ids
(apt-cache, apt-lists x4 RUN blocks, npm-cache x2, next-cache x1) with
the omni-route service id.
* fix(sse): remove duplicate sseCommentsEnabled import in stream.ts
Turbopack rejected the file with 'the name sseCommentsEnabled is
defined multiple times' — imported once at the top of the file and
again lower down from the same module. Broke every production build
(Docker/Railway) at the release/v3.8.50 tip, independent of the cache
mount fix in this branch. Validated by a full Docker build on Railway
completing past this step.
* fix(sse): answer tiny-budget reasoning probes with a truncated 200 (#10281)
Claude Code's /model capability check sends max_tokens: 1. Reasoning
models burn the whole probe on thinking, and some upstreams (e.g.
api.cline.bot for deepseek-v4-flash) answer the empty outcome with a
5xx "empty response content" instead of a truncated 200. The relayed
failure also marked the connection unavailable and poisoned
fallback/cooldown bookkeeping for what is only a probe.
Detect tiny-budget reasoning probes in the non-streaming providerFailure
path and synthesize a valid truncated response (200, empty content,
finish_reason "length") — the same semantics errorClassifier.ts already
grants to length-truncated empty 200s. Probes no longer poison
connection health. Refs #10281.
* chore(changelog): add fragment for reasoning-probe truncated-200 fix (#10284)
sql.js has no incremental write path, so persist() rewrites the whole image on
every save. Going through fs.writeFileSync(filePath, ...) opened the destination
with O_TRUNC, leaving the on-disk database 0 bytes and then partial for the whole
write -- a window that scales with database size and recurs on every save.
Unlike better-sqlite3 / node:sqlite, that window is not covered by SQLite's
locking protocol, so it is visible to every other process reading the same file:
a backup job, a metrics exporter, an operator running sqlite3. Those readers get
SQLITE_CORRUPT ("database disk image is malformed") while PRAGMA
integrity_check passes moments later, which makes the failure look random and
blames the reader.
Now: temp file in the same directory, fsync, rename() over the destination.
rename is atomic on POSIX and on Windows for a same-volume replace, so a reader
sees either the previous image or the new one, never a truncated one. It also
closes a total-loss window: a crash mid-write used to leave the real database
truncated, and now only leaves a stale temp file behind.
The regression guard asserts the property that separates the two implementations
without racing a timer: a reader that opened the file before a save still reads a
complete, valid image afterwards, and the published file sits on a new inode.
It fails on the previous implementation and passes on this one.
Co-authored-by: Max <maxmad64@gmail.com>
Co-authored-by: adevwithpurpose <adevwithpurpose@users.noreply.github.com>
* fix(providers): make the monsterapi deprecation from #8676 actually apply
#8676 marked MonsterAPI deprecated after its domain stopped resolving, but
wrote the flag as `isDeprecated`. Nothing reads that key. The field the
codebase consumes is `deprecated`:
src/shared/validation/providerSchema.ts declares `deprecated`
ProviderCard.tsx strikethrough + block icon + reason
ProviderTestSlideOver.tsx warning
providerOnboardingCatalog.ts Boolean(provider.deprecated), sorts last
ProviderOnboardingWizard.tsx deprecated badge
scripts/docs/gen-provider-reference.ts gates the DEPRECATED note
Zod object schemas ignore undeclared keys, so `isDeprecated` never failed
validation - it was dropped silently. The deprecation therefore had no effect
anywhere, and tests/unit/8676-monsterapi-deprecation.test.ts asserted the same
unread key, so it stayed green while guarding nothing.
The committed docs/reference/PROVIDER_REFERENCE.md is the visible proof: the
generator renders predibase (which uses `deprecated`) with a DEPRECATED note,
while monsterapi still advertised "Get API key at monsterapi.ai" - a domain
that does not resolve (probed 2026-08-13: api.monsterapi.ai and monsterapi.ai
both 000, against api.openai.com 401 as a reachability control).
Rename the key, repair the regression test to assert the consumed field and to
reject the undeclared one, and refresh the generated reference row.
* fix(providers): name the changelog fragment for PR #10234
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: pacocartones <pacocartones@users.noreply.github.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Co-authored-by: adevwithpurpose <adevwithpurpose@users.noreply.github.com>
Present xAI API-key and OAuth connections through one dashboard card while preserving the distinct backend IDs required for refresh and quota handling.
Co-locate both registry entries and include canonical and legacy connection IDs in provider fetch and batch-test flows.
* feat(providers): add local ZCode ACP backend
* test(snapshots): regenerate translate-path golden for zcode provider
The new local ZCode ACP backend (zcode://app-server/stdio) was added to the
provider catalog but the translate-path golden snapshot was not regenerated,
so the combined suite (provider-translate-path-golden.test.ts) failed on the
merged tip. Regenerate the snapshot to include the zcode translate-path entry.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* docs(env): document ZCODE_* vars for the local zcode provider
Registers the 11 ZCODE_* env vars read by the zcode executor (.env.example
+ docs/reference/ENVIRONMENT.md) so the env-doc-sync gate stays green.
Co-authored-by: Diego Souza <8016841+diegosouzapw@users.noreply.github.com>
* test(autoCombo): include zcode in the glm-family provider set
#10184's local zcode backend advertises the full GLM_SHARED_MODELS
line-up (registry/zcode, authType none) — same documented case as auggie
and devin-cli-agentic. Update auto/glm provider-set assertion to include
it.
Co-authored-by: Diego Souza <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: roomhacker <roomhacker@bezrabotnyi.com>
Co-authored-by: adevwithpurpose <adevwithpurpose@users.noreply.github.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
resolvePortPid shelled out to lsof alone. On a host without it, spawn
raises ENOENT, the error handler turned that into null, and the caller
could not tell 'nothing holds this port' from 'I have no way to look' -
so a service adopted on a supervisor restart kept pid: null forever,
silently, which is the regression the adopt-branch test guards against.
Probes lsof, then ss, then netstat, sharing one deadline so the whole
lookup still costs at most PID_RESOLVE_TIMEOUT_MS. Output parsing for
each is a pure exported function so the formats are unit-testable
without the binary being installed.
netstat cannot filter by port, so its parser matches the local-address
column rather than scanning the line, keeping a foreign address that
ends in the same number from being read as a listener.
Implements the secure, opt-in Video Bridge for issue #9760, including bounded FFmpeg frame extraction, capability-aware routing, telemetry, settings UI, localization, documentation, and regression coverage.
* fix(sse): let :free OpenRouter models bypass connection-wide credits_exhausted lock
A 402 from one paid OpenRouter model correctly locks the whole connection
as credits_exhausted for an hour (intentional, per #6842), but that lock
was also blocking every :free model on the same connection even though
OpenRouter bills free models separately from account credits.
Reconstructed clean against release/v3.8.50 by the maintainer: the author's
original branch predated a large auth.ts import refactor; the same delta was
re-applied onto the current tip and the TDD test still passes.
TDD: tests/unit/openrouter-free-model-credits-exhausted.test.ts
reproduces the bug (fails before the fix, passes after) and covers the
three guard cases above.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* test(mutation): register openrouter-free-model-credits-exhausted in stryker tap.testFiles
The new unit test covers src/sse/services/auth.ts, which is one of the 31
stryker-mutated modules — per check-mutation-test-coverage every covering
test must be listed in tap.testFiles or its mutant kills stop counting.
Registered the file so the blocking mutation-test-coverage gate passes.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: killmonger2317-coder <282069920+killmonger2317-coder@users.noreply.github.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(kilocode): strip unsupported response_format for DeepSeek (400 regression)
kilocode's DeepSeek V4 Flash rejects ANY response_format — both
json_schema AND json_object 400 with 'Invalid input: response_format'
(verified live 2026-08-15 via the Hindsight fact-extraction path on
kilocode/deepseek/deepseek-v4-flash). The default executor's
applyJsonSchemaFallback only covered openai-compatible-* providers and
only downgraded json_schema -> json_object, so kilocode forwarded the
unsupported format raw. Same bug class as the opencode fix#9992.
For kilocode: strip response_format entirely and inject the schema (or a
plain 'valid JSON only' instruction for json_object) into the system
prompt. openai-compatible-* keeps the existing json_schema downgrade and
json_object passthrough (they accept both).
Regression tests: kilocode json_schema is stripped + schema-injected;
kilocode json_object is stripped + JSON-only instruction; both verified
to fail without the fix (sabotage: 2 fail). All 49 executor-default-base
tests pass.
* fix(kilocode): drop as-any casts in new tests to clear the frozen ESLint baseline
The file's frozen no-explicit-any baseline is count 42; the new kilocode
strip tests added 3 net-new 'as any' casts, tripping the --max-warnings 0
lint-guard. Replace them with typed assertions that carry the same checks.
Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
---------
Co-authored-by: benzntech <benzntech@users.noreply.github.com>
Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
The upstream Messages API rejects directive-style messages (empty content
array with a message-level output_config) when they sit at messages[0] —
the initial system prompt position — while accepting the form at any other
position. Measured in production: 122x 400 on the offical-claude combo in
one hour.
The mid-conversation-system passthrough (official provider + 1M-context
beta models) keeps system-role messages inside messages[], so a directive
that arrived first went upstream unchanged. relocateDirectiveOnlyMessages()
moves the whole leading run of empty system messages: directive-only ones
past the first real turn, plain empties dropped. extractSystemRoleMessages()
now folds a directive's output_config into the top-level parameter instead
of silently discarding it.
Signed-off-by: Minxi Hou <houminxi@gmail.com>
Auto combos (virtual auto/* pools via virtualFactory and pure-auto named
combos via expandAutoComboCandidatePool) expanded their candidate pool from
the provider's STATIC registry catalog, which can include models the operator
never synced or approved (e.g. openrouter/auto). The visibility filter
(getHiddenModelsByProvider) only caught models explicitly flagged isHidden,
so catalog-only models passed through and got routed upstream.
Build the credentialed pool from the models the user actually has available
(synced + custom non-hidden), falling back to the static catalog only when
the operator has no synced/custom models for that provider. Applies to every
provider uniformly (openai, kilocode, openrouter, ...), with per-connection
scoping for synced models. Provider wildcards (providerWildcard.ts) already
used the active synced catalog as the authoritative source.
Regression coverage: tests/unit/combo-auto-pool-visible-only.test.ts
Co-authored-by: herjarsa <herjarsa@users.noreply.github.com>
* feat(providers): add tencent-aistudio-web cookie provider (tasw)
* fix(sse): remove orphaned DevinDesktopExecutor import from executor index
The "devin-desktop" executor key is unused (devin-desktop provider config
resolves to executor "devin-cli"); the imported ./devin-desktop.ts file
was never present, so executors/index.ts failed to load (ERR_MODULE_NOT_FOUND)
and broke every unit test that imports the executor registry (e.g.
tests/unit/deepseek-web.test.ts). Stale base sync carried this into the branch.
Remove the dead import/registration/export.
* fix(providers): restore DevinDesktopExecutor registration in executor index
The previous commit removed the devin-desktop executor import/registration/
export from open-sse/executors/index.ts, but the devin-desktop provider
registry still resolves executor "devin-desktop" and
tests/unit/devin-providers.test.ts asserts hasSpecializedExecutor("devin-desktop")
is true. The removal broke 6 tests in that file. Restore the three lines so
the live Devin Desktop executor keeps serving the provider.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(providers): correct tencent-aistudio-web wrapper shape + provider count sync
Return {response,url,headers,transformedBody} instead of a raw fetch Response
(the executor contract every other executor in this file follows) and
re-wrap the upstream body so it uses the local Response constructor, not the
undici-patched one from globalThis.fetch.
Regenerate docs/reference/PROVIDER_REFERENCE.md and sync the 339->340
provider-count claims (README, AGENTS.md, llm.txt + 42 i18n mirrors,
package.json, promise-pillars/comparison-table/cli-terminal SVGs) that this
PR's new provider invalidated.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* docs(providers): sync readme-hero.svg provider count claim (339->340)
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(providers): register tencent-aistudio-web web-session credential metadata + golden
Add the WEB_SESSION_CREDENTIAL_REQUIREMENTS entry for tencent-aistudio-web
(cookie-based, matching the executor's raw Cookie-header credential) and
regenerate the translate-path golden snapshot to include the new provider —
both were failing CI unit tests that enumerate every registered provider.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(providers): align tencent-aistudio-web test with the wrapper-shape contract
The test asserted res.status/res.json() directly against executor.execute()'s
return value, matching the pre-fix (broken) raw-Response shape. Update it to
read res.response.status/res.response.json() — the {response,url,headers,
transformedBody} contract every executor in this codebase follows.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: MeRezaRezaei <MeRezaRezaei@users.noreply.github.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Co-authored-by: adevwithpurpose <adevwithpurpose@users.noreply.github.com>
Deploying the internal gateway was a manual build/pack/scp/npm-i/pm2-restart sequence with no record of what landed and no proof it served traffic. On 2026-08-14 that shipped a package built from a branch predating #10373: the process came up, health said 'healthy', and every request returned 502 until a human hit it.
scripts/ops/deployCanary.ts holds the policy as pure functions — refuse an artifact that is not traceable to the release line (reusing #10427), and grade the deploy on health PLUS at least one real completion. Zero probes fails: 'no probe ran' must never read as 'everything is fine', which is exactly how a broken egress path hides behind a green health check. Remote steps are argv arrays, never shell strings (Hard Rule #13), ordered so the rollback anchor is captured before the install overwrites it.
scripts/ops/deploy-canary.mjs performs the side effects, supports --dry-run, and prints the rollback command when the smoke fails.
Closes#10429
The packaged artifact stamped dist/BUILD_SHA but nothing verified the SHA belonged to the release line, so a tarball built from a feature branch installed and served traffic indistinguishably from a release build. That is how the internal gateway ended up running a build that predated #10373 and answered every request with 502 'Executor result must contain a Response' — identifying it required SSH plus grepping the compiled chunks.
scripts/build/buildProvenance.ts classifies a build SHA against the release ref (pure functions, injected git probe). A missing SHA fails even with the canary override: an unidentifiable artifact cannot be vouched for. validate-pack-artifact enforces it on real packs (skipped under --policy-only, which runs without a build); OMNIROUTE_ALLOW_CANARY_BUILD=1 records a deliberate off-release-line build instead of failing it. /api/monitoring/health now exposes system.buildSha — absent when unknown, never fabricated.
Closes#10427
Any process that opened the DB without setting DATA_DIR resolved to ~/.omniroute/storage.sqlite — the operator's live database, provider credentials included. tests/_setup/isolateDataDir.ts only covers the npm scripts; the documented single-file test command and ad-hoc probes bypassed it (one did exactly that during #10334).
resolveWritableDataDir now redirects a test-context process with no DATA_DIR to a throwaway temp dir, stable per process. Redirect rather than throw, so the documented single-file command keeps working; OMNIROUTE_ALLOW_DEFAULT_DATA_DIR=1 opts back in and records the intent.
Closes#10428
Audit of every numeric claim in the README, AGENTS.md and the README SVGs against
live code, plus a changelog/credit reconciliation over the full v3.8.50 cycle.
Corrected numbers (all measured, not estimated):
- Provider circuit breaker thresholds were scaled up in code for 500+ connection
deployments (`providerFailureThreshold`: OAuth 3 -> 10, API key 5 -> 15) but the
docs still published the pre-scale values. Fixed in AGENTS.md (now a table that
also separates the provider-level threshold from the per-connection one and lists
the provider cooldowns), in the README alt text and inside resilience-layers.svg
(visible label and aria-label).
- "40+ free forever" was unsourced. Measured from the free-tier catalog as every
provider whose free access renews or needs no key (recurring-monthly, -daily,
-uncapped, -credit, keyless; one-time signup credits and discontinued pools
excluded): 56. Updated in the README and promise-pillars.svg.
- Cycle-evolution table: v3.8.49 shipped 290 providers, not 291, and the model row
compared the v3.8.49 free-tier catalog (516) against today's full catalog. Both
columns now use the same metric - distinct documented models, 1185 -> 1202.
- Tech-stack row: 95 domain modules -> 117.
The free-forever count is now enforced by check:docs-counts so it cannot drift
again; it is derived from freeType in the live catalog, like every other gated
number.
Changelog reconciliation (`scripts/release/list-uncovered-commits.mjs`):
uncovered cycle commits drop from 149 to 62. 75 user-facing commits gained a bullet
with author attribution, the 45 ref-less direct pushes and 29 chore/ci/test/docs
commits were consolidated into rollup bullets, and the contributors table grew from
147 to 161 rows - 14 contributors who had landed work with no credit at all
(including @amartinawi, @pacocartones and @excessivechaos) are now credited.
The remaining 62 carry no PR/issue ref, which is the ceiling of ref-based coverage.
CHANGELOG.md and its i18n mirrors are added to .prettierignore: check:changelog-
integrity compares base bullets as exact strings, and Prettier normalizes markdown
emphasis inside them (*from* -> _from_), so any PR that staged the changelog turned
the merge-integrity job red. scripts/release/* is the changelog's formatter of
record, the same precedent already used for ENVIRONMENT.md and PROVIDER_REFERENCE.md.
Co-authored-by: Xiangzhe <bakryun0718@proton.me>
* docs: add rate limiting guide for free providers (429/400/401)
Community-reported troubleshooting for auto-discovered issues when
rotating through free/no-auth providers (opencode, felo-web, auggie).
Documents the verified env-var combo that eliminates
intermittent 429/400/401 failures in cron/agent automation:
OMNIROUTE_ROTATE_ON_400=true,
OMNIROUTE_CHAT_MAX_HEAVY_IN_FLIGHT=4,
OMNIROUTE_STRUCTURE_LIMIT=off
Includes root-cause breakdown (provider quota vs passthrough 401 vs
concurrency amplification), verification steps via /monitoring/health,
and escalation for hard quota exhaustion.
* docs(providers): fix fabricated env var and breaker states in rate-limit guide
Replace OMNIROUTE_STRUCTURE_LIMIT (does not exist in the codebase) with
OMNIROUTE_CHAT_ADMISSION_QUEUE_MS and document the real rate-limit knobs
(RATE_LIMIT_MAX_WAIT_MS / RATE_LIMIT_MAX_QUEUE_DEPTH / RATE_LIMIT_AUTO_ENABLE).
Correct the circuit breaker states to the actual enum (CLOSED/DEGRADED/OPEN/HALF_OPEN)
and point the health-check note at circuitBreakers.providerBreakers[].state.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: Bruno <bruno@nousresearch.com>
Co-authored-by: mrcram2021 <mrcram2021@users.noreply.github.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Co-authored-by: adevwithpurpose <adevwithpurpose@users.noreply.github.com>
* fix(db): prune pre-migration backups so db_backups stops growing unbounded
createPreMigrationBackup() wrote a VACUUM INTO snapshot on every migration run
and never pruned. On a long-lived instance db_backups/ reached 48.999 files /
204 GB against a 5,3 MB live database; a second devbox showed the same shape
(5.711 files / 24 GB).
The retention policy already existed in cleanupDbBackups() but nothing on the
migration path reached it — its only callers are backup.ts and the
/api/db-backups route, neither of which runs during a migration.
migrationRunner.ts cannot import backup.ts: core.ts imports migrationRunner.ts
and backup.ts imports core.ts, so that edge would close a cycle. The policy
therefore moves to a new core-free module, backupRetention.ts, which both call
sites share — cleanupDbBackups() now delegates to it rather than duplicating it.
At the migration call site the operator's maxFiles/retentionDays are read
through the adapter already open for the run; going through getDbInstance()
would re-enter database initialization. Pruning never throws, so housekeeping
cannot fail a migration.
Closes#10421
* chore(db): declare backupRetention as an intentionally-internal db module
check:db-rules requires every src/lib/db/ module to be either re-exported by
localDb.ts or listed in INTENTIONALLY_INTERNAL. backupRetention.ts is a shared
primitive consumed only by db/backup.ts and db/migrationRunner.ts — the same
category as the migrationRunner entry — so it belongs in the allowlist rather
than in the public re-export surface.
* test(db): include backupRetention in the audited INTENTIONALLY_INTERNAL list
check-db-rules-classification.test.ts freezes the exact membership of
INTENTIONALLY_INTERNAL, so adding the 40th entry has to be reflected there too
— the gate script and this test pin the same contract from opposite sides.
---------
Co-authored-by: Xiangzhe <bakryun0718@proton.me>
i-have-adhd shipped in #10271 with en + pt-BR only; it now covers vi/ja/id as well,
matching ponytail. Every level keeps the SHARED_BOUNDARIES clause.
Adds a per-MATRIX guard (output-styles-i18n-matrix.test.ts). Every existing test is
per-style, which is how less-code stayed English-only since the 9router port without
anyone noticing. The guard fails on: a new style without pt-BR, a style losing a
translation it had, a translation missing an intensity level, a translation that dropped
the boundaries clause, and a stale KNOWN_ENGLISH_ONLY entry.
Proven by mutation, not just by passing: dropping less-code from the allowlist and
renaming the vi key both turned it red with the expected messages.
less-code stays English-only as declared debt (KNOWN_ENGLISH_ONLY + comment); the
remaining coverage work is tracked in #10426.
Makes the ProviderErrorRule `scope` field real at the persistence layer, exclusively for agentrouter (owner decision; every other provider keeps byte-identical behavior).
checkFallbackError now surfaces `ruleScope` behind the HONORS_RULE_LOCK_SCOPE_PROVIDERS allowlist, and the agentrouter 403 path consults the rules before the generic apikey-FORBIDDEN early-return. markAccountUnavailable honors scope "connection" with a temporary connection cooldown instead of a per-model lockout — guarded so a permanent state can never be downgraded to a transient retry loop — and combo now skips the exhausted account within the same request, which also stops force-reusing the just-cooled connection via allowRateLimitedConnection.
Documented in RESILIENCE_GUIDE §7 with the honest limits (disableCooling connections keep per-model behavior; the 6h model-access cooldown is clamped by mlSettings.maxCooldownMs, 30min by default; same-request skip needs targets carrying their own connectionId).
Closes#10334
* fix(ci): pin Build (advisory) to a hosted runner with memory provisioning
`Build (advisory)` has been reporting a permanent red on every PR while
producing no usable signal at all.
Measured over the last 25 quality.yml runs (2026-08-14): not one instance of
the job reached a conclusion. Every sample was either queued on the
self-hosted pool — 2 runners, omniroute-113-6/7, both permanently busy; one
job sat queued for over 2 hours and was still unclaimed — or, when it did land
on a runner, killed mid-build by this workflow's own cancel-in-progress
concurrency. All 6 sampled "failures" are exit 143 / "The runner has received
a shutdown signal" at ~3.5 min into `npm run build`. Zero OOM, zero build
errors. The job was consuming a runner the real gates compete for while
telling every PR author it was broken.
Gap 19 deliberately left USE_VPS_RUNNER governing build-like jobs, on the
premise that the build needs the .113's RAM. That premise no longer holds:
`Fast Production Build` (build.yml) runs `build:release` — a superset of this
job's `npm run build`, plus the CLI bundle — on plain ubuntu-latest and passed
24 of its last 25 runs in ~15 min. The difference is memory PROVISIONING, not
the machine: a 10 GB swapfile plus a 12 GB V8 heap. Swap is the part that
matters, because --max-old-space-size bounds only V8's JS heap and never
Turbopack's native Rust allocation (#6409).
Pins the job to ubuntu-latest and mirrors both settings from build.yml.
USE_VPS_RUNNER keeps its other consumers (ci.yml Build, nightly-release-green,
npm-publish), so the variable stays meaningful. Fork safety is strictly
improved: no PR can reach the LAN runner through this job any more.
check:workflows --ratchet: 186 zizmor findings, baseline 190, no regression.
prettier + YAML parse: clean.
* fix(ci): scope Build (advisory) to fork PRs
Follow-up to the hosted-runner pin in this same PR, after measuring what the
job is actually for.
build.yml's `Fast Production Build` triggers on `push: branches: ["**"]` and
runs `build:release` — a superset of this job's `npm run build`, plus the CLI
bundle. For an own-origin branch that push fires here, so the tree was being
built twice per PR. A fork contributor pushes to THEIR repo, so build.yml
never runs in this repo and this job is their only pre-merge build signal.
That could have argued for deleting the job, except the traffic says
otherwise: 72 of the last 100 PRs into release/** come from forks. The fork
case is the majority, not the exception. So the job earns its place — it just
should not duplicate build.yml for the own-origin 28%. Added the fork filter
to the existing `if`.
Also corrects the reliability claim in the previous commit message. Over a
wider window the job is not literally never-green: across 2026-08-13/14 it
reached `success` on roughly 10-15% of runs (13/138 on 08-14, 7/53 sampled on
08-13). Chronically unreliable, not permanently dead — the conclusion and the
fix are unchanged.
The #7307 guard in tests/unit/build/check-workflows.test.ts pinned the old
self-hosted expression, so it is realigned here: it now asserts the hosted
pin, the absence of self-hosted/USE_VPS_RUNNER in the job's DIRECTIVES (the
comment legitimately explains why the pool was abandoned, so the scan strips
comments), both memory settings, and the fork filter. Mutation-validated —
restoring self-hosted, dropping the swapfile, or flipping the fork filter each
turns it red.
check-workflows.test.ts: 32 pass, 0 fail.
check:workflows --ratchet: 186 findings, baseline 190, no regression.
---------
Co-authored-by: Xiangzhe <bakryun0718@proton.me>
