Phase 0a reconciliation (/generate-release): every commit since v3.8.44 now
has a bullet or a Maintenance rollup (82 commits; the only ref-less residues
are the bump/recording commits); #6193 bullet gains its PR ref; #6298
diagnosis credited (@subhansh-dev, landed via #6234); [3.8.44] header dated;
'### 🙌 Contributors' table injected (27 external + maintainer) + 42 i18n
mirrors resynced.
* ci(vps): extend the dynamic omni-release runner to e2e/integration/electron + nightly pre-flight
Completes the #6284 rollout to the jobs where the VPS pays the most:
- test-e2e (9 shards, 15-20min/shard hosted — dominated by setup, not tests),
test-integration (2 shards) and electron-package-smoke now pick the
self-hosted omni-release runner under the same gate: vars.USE_VPS_RUNNER
== 'true' AND own-origin (fork PRs never reach self-hosted).
- nightly-release-green (the release pre-flight) becomes runner-dynamic too:
on the VPS it runs in a clean env — no operator OMNIROUTE_API_KEY, no
local noauth CLIs — eliminating the machine-specific false positives that
dominated the 2026-07-05 pre-flight; passes --hermetic (no-op until the
#6300 validator lands, then belt-and-suspenders).
Validation plan (per operator request): release-runner-up.sh -> full ci.yml
workflow_dispatch on this branch exercising e2e/integration/electron on the
VPS (proves playwright --with-deps + xvfb on the runner) -> down.sh -> VM
off verified.
* fix(mitm): test suite and CI must never mutate the OS trust store (OMNIROUTE_SKIP_SYSTEM_TRUST)
Incident 2026-07-05 on the self-hosted release runner (VM 113): the
integration test 'POST /cert: installs trust when cert exists' exercised the
REAL install path, wrote a 105-byte fake PEM (FakeMITMCertForTestingOnly)
into /usr/local/share/ca-certificates and update-ca-certificates baked the
invalid entry into ca-certificates.crt — breaking ALL system TLS on the VM
(curl error 77, apt cert failures, and the intermittent gzip-corrupted
next-build artifacts that failed 6/9 e2e shards in run 28754447912). Hosted
runners are ephemeral, so the same mutation went unnoticed for months.
- installCert/uninstallCert: skip the OS dispatch under
OMNIROUTE_SKIP_SYSTEM_TRUST=1 — AFTER the input checks, so the #4546
environment-skip contract (missing file throws -> structured skip) and the
already-installed/not-installed early returns are preserved.
- installTproxyCa/uninstallTproxyCa: same guard, only when no run dep is
injected (DI'd tests keep exercising the full command sequence with mocks).
- tests/_setup/isolateDataDir.ts sets the env for every node:test process;
ci.yml/quality.yml/nightly-release-green.yml set it workflow-wide (e2e runs
the real app outside the test setup).
TDD: tests/unit/system-trust-test-guard.test.ts (guard exported to every test
process; guarded install resolves on a real file without touching the OS;
missing-file contract preserved). 82/82 across the affected cert/tproxy/
agent-bridge suites.
* ci(vps): descope — e2e/integration/electron stay on hosted runners; keep the hermetic nightly pre-flight dynamic
Validation verdict (runs 28754447912 + 28757670732, VM 113):
- e2e cannot run >1 per VM: both jobs bind port 20128 ('already used') — needs
a per-job port in the playwright runner before any VM rollout.
- concurrent ~1GB artifact downloads truncate on the VM uplink (gzip 'invalid
compressed data' — with 2 runners the e2e shard passed; corruption returned
at 4) — actions/download-artifact has no integrity retry here.
- integration shard 2 exceeded its 15-min timeout twice on the VM.
The VPS remains a win for whole-machine jobs: build/unit/vitest (already
dynamic via #6284) and nightly-release-green (single job, clean env, hermetic
pre-flight) — which this PR keeps.
Incident 2026-07-05 on the self-hosted release runner (VM 113): the
integration test 'POST /cert: installs trust when cert exists' exercised the
REAL install path, wrote a 105-byte fake PEM (FakeMITMCertForTestingOnly)
into /usr/local/share/ca-certificates and update-ca-certificates baked the
invalid entry into ca-certificates.crt — breaking ALL system TLS on the VM
(curl error 77, apt cert failures, and the intermittent gzip-corrupted
next-build artifacts that failed 6/9 e2e shards in run 28754447912). Hosted
runners are ephemeral, so the same mutation went unnoticed for months.
- installCert/uninstallCert: skip the OS dispatch under
OMNIROUTE_SKIP_SYSTEM_TRUST=1 — AFTER the input checks, so the #4546
environment-skip contract (missing file throws -> structured skip) and the
already-installed/not-installed early returns are preserved.
- installTproxyCa/uninstallTproxyCa: same guard, only when no run dep is
injected (DI'd tests keep exercising the full command sequence with mocks).
- tests/_setup/isolateDataDir.ts sets the env for every node:test process;
ci.yml/quality.yml/nightly-release-green.yml set it workflow-wide (e2e runs
the real app outside the test setup).
TDD: tests/unit/system-trust-test-guard.test.ts (guard exported to every test
process; guarded install resolves on a real file without touching the OS;
missing-file contract preserved). 82/82 across the affected cert/tproxy/
agent-bridge suites.
Schemathesis's newer unsupported-methods check (unpinned tool drift) sends
QUERY /api/keys/{id}/devices and demands 405 Method Not Allowed; the path had
no HIGH_RISK_METHOD_RULES entry, so the auth layer answered 401 first. Add
the devices rule (GET-only) so undocumented methods get a clean method-first
405 — same pattern as the v3.8.44 TRACE fix. TDD:
tests/unit/dast-method-not-allowed.test.ts gains the devices QUERY case (4/4).
- check:error-helper: githubSkillTools.ts (#6186 wave) built MCP install
error results with raw err.message — routed through sanitizeErrorMessage()
(Hard Rule #12; 19/19 agentSkillTools tests green)
- check:mutation-test-coverage: tests/unit/combo-provider-cooldown-sibling.test.ts
(#6216) was missing from stryker.conf tap.testFiles — added so its mutant
kills count on nightly-mutation
Three latent heavy-CI reds surfaced by the VPS validation dispatch (the fast
path never runs these gates):
- t06 route-validation: POST /api/github-skills destructured request.json()
blind — a non-array 'targets' would .map-crash. Now validateBody(zod)
with defaults preserved (Hard Rule #7). Guard:
tests/unit/github-skills-route-validation.test.ts (4/4).
- env-doc-sync: document OMNIROUTE_SKIP_SYSTEM_TRUST (#6310) and the
changelog-integrity gate envs CHANGELOG_BASE_REF/ALLOW_CHANGELOG_REMOVALS
(#6300) in .env.example + ENVIRONMENT.md.
- zizmor ratchet: the new merge-integrity job's checkout/setup-node uses were
unpinned (+1 finding, 160 > baseline 159); pinned by SHA -> 158 (< baseline).
#6292 rewrote/added zero-width-marker tests with 7 new 'as any' result casts,
pushing the file past its frozen suppression (84) — and when violations
exceed the suppressed count ESLint reports ALL of them, so the ci.yml Lint
job went red with 90 errors. The 7 new sites get typed shapes (delta /
arguments / choices / output accessors — same pattern as fecf888fd); the
pre-existing debt stays frozen at the exact new count (83). 50/50 tests
green, file lint-clean under the suppressions baseline.
#6186 added omni-github-skills (API 22 -> 23) and updated computeCoverage's
total, but left the old count hardcoded in the A2A layer: listCapabilities
metadata reported coverage.api.total 22 (type literal + value) and
SkillCoverageSchema pinned z.literal(22) — so the schema would REJECT the
correct runtime value. Aligned all three to 23 + the unit fixtures
(listCapabilities-a2a, agentSkills-schemas, 46/46 with agentSkillTools-mcp).
Merged into release/v3.8.45. CI merge-integrity fast-gates (changelog-integrity + agent-skills-sync) + pre-flight hermetic mode. O gate novo detectou 11 SKILL.md gerados fora de sync (drift pré-existente no tip: omni-api-keys endpoints, omni-github-skills do #6186) — regenerados neste PR, Merge-integrity GREEN no CI. Reds restantes base-red (dast-smoke/file-size drift/live-data flaky). Testes novos 17/17.
Merged into release/v3.8.45. Delta do par #6246 (Test-All read-only + bulk enable/disable), reconciliado com o núcleo #6296 já mergeado. CHANGELOG restaurado (ambos bullets do #6246 coexistem). Reds pré-existentes: dast-smoke (infra) + file-size drift.
Merged into release/v3.8.45. Reds pré-existentes classificados: dast-smoke (infra), check:file-size (drift de baseline de god-files congelados), e 3 testes de contagem agentSkills stale (43→44 já corrigidos no tip pelo #6186 — somem no squash sobre o tip). Núcleo do fix#6246 (proxy IP-leak + over-deactivation).
Entries whose violations no longer exist (cleaned by cycle merges and the
pre-flight fixes) made 'npm run lint' exit 2 with 'suppressions left that do
not occur anymore'. Regenerated via --prune-suppressions; net-new policy
unchanged.
The 'streaming no recognized content' branch added by #6216 marked ANY
stream that ended without content deltas as invalid — sweeping in two
regression-guarded pass-through contracts: an empty stream terminated by an
explicit 'data: [DONE]' (#3399 context-cache protection) and an incomplete
Claude lifecycle (ping only, no message_start; #3685 — stream-readiness
timeout territory, not failover). Both unit guards were red on the branch
and green on main (86/86 vs 84/86).
The branch now fires only for a truly EMPTY body (zero bytes — the Gemini
HTTP-200-empty case that motivated #6216), tracked via sawAnyBytes. New
guard: '#5976 truly EMPTY streaming body (zero bytes) -> invalid for combo
failover'. 87/87 across both files.
Also in this pre-flight batch:
- agentSkillTools-mcp: api.have upper bound 22 -> 23 (the #6186 catalog
addition updated the totals but missed this bound)
- delete tests/unit/free-provider-rankings-configured-filter.test.ts:
#6251 (server-side configuredOnly/availableOnly) superseded the #6245
client-side toggle it pinned; replacement declared in the test-masking
allowlist (tests/unit/freeProviderRankings-filters.test.ts, 11/11)
PR #6186 added omni-github-skills to the agent-skills catalog (API 22 -> 23)
but did not run the generator, so skills/omni-github-skills/SKILL.md never
existed and 6 integration assertions split between the old (42/43) and new
counts. Generated via scripts/skills/generate-agent-skills.mjs --apply and
aligned agent-skills-discovery to the real totals (43 = 23 API + 20 CLI;
handlers return 44 with config-codex-cli). 30/30 discovery+content tests green.
- executor-kiro/save-call-log/call-logs-correlation tests: replace 15 'as any'
casts with typed shapes (net-new no-explicit-any errors from #6213/#6216);
prune the now-empty suppression entries so the frozen baseline stays exact
- github-skills + usage/call-logs routes: raw toLowerCase().includes() search
replaced by matchesSearch() (no-restricted-syntax — Turkish-safe search,
behavior covered by tests/unit/call-logs-correlation-substring.test.ts and
tests/unit/github-collector.test.ts)
- validate-release-green.mjs: run ESLint with --suppressions-location (match
the npm run lint contract — frozen debt is not a release red) and raise the
lint timeout 15->30min (a full pass takes ~14min alone; the 15min ceiling
expired under concurrent suite load and surfaced as 'could not parse eslint
json')
- test-masking: allowlist the 4 verified-legitimate assert reductions of the
cycle (#6248 MiMo V2 removal, #6170 Kiro catalog correction, #6154 Copilot
catalog refresh) and register the #6164 AutoRoutingBanner test deletion with
a real replacement guard (tests/unit/home-no-autorouting-banner.test.ts —
asserts the banner stays out of home/page.tsx and the component stays deleted)
- docs-sync: executors count 68 -> 73 in ARCHITECTURE.md + CODEBASE_DOCUMENTATION.md
- env-doc-sync: document OMNIROUTE_NO_SUDO (#6249/#6122) in .env.example +
docs/reference/ENVIRONMENT.md
Adds the on-demand self-hosted runner plumbing for /generate-release:
- scripts/vps/release-runner-up.sh: starts the runner VM on Proxmox, waits
for >=1 'omni-release' runner to report online via the GitHub API, then
flips the USE_VPS_RUNNER repo variable to true. Any failure/timeout sets
it back to false and exits 1 so the caller falls back to hosted runners.
- scripts/vps/release-runner-down.sh: flips USE_VPS_RUNNER=false FIRST
(so no job gets scheduled onto a dying runner), then gracefully shuts
the VM down. Idempotent.
- ci.yml: build, test-unit x8 and test-vitest pick their runner
dynamically. Self-hosted is used ONLY when USE_VPS_RUNNER == 'true'
AND the event is own-origin (push/dispatch, or a PR whose head repo is
this repository). Fork PRs and the var's default/absent state always
fall back to ubuntu-latest.
Why: the Free plan caps hosted concurrency at 20 jobs; a release run
saturates it and queues. The benchmarked VPS (32-core, 4 runners) matches
hosted per-job times (unit ~8.5min/shard, build 9min turbo) but eliminates
the queue, which is the real release bottleneck (~30-50min on a busy day).
Scope is conservative: only the three job families benchmarked on the VPS;
e2e/electron/integration stay hosted (playwright/xvfb provisioning not
validated on the runner workspace yet).
Flips the builder stage to OMNIROUTE_USE_TURBOPACK=1 and rewrites the stale
panic comment: the v3.8.27-era TurbopackInternalError ('entered unreachable
code: there must be a path to a root' in ImportTracer::get_traces) no longer
reproduces on Next 16.2.9.
Validation (2026-07-05, this exact Dockerfile, default
OMNIROUTE_BUILD_MEMORY_MB=4096, no overrides):
- amd64: docker build 659s, exit 0, zero panic/OOM strings in the full log,
container smoke-tested (/api/monitoring/health 200)
- arm64 (qemu): exit 0, zero panic strings
Webpack stays available as the escape hatch:
--build-arg / -e OMNIROUTE_USE_TURBOPACK=0. The V8 heap ceiling is kept:
Turbopack's compile is native Rust, but prerender/export still runs on V8.
Turbopack (stable in Next 16) becomes the code default in the three entry
points that previously required an explicit OMNIROUTE_USE_TURBOPACK=1:
- scripts/build/build-next-isolated.mjs (production build)
- scripts/dev/run-next.mjs (dev server)
- scripts/dev/run-next-playwright.mjs (playwright dev runner)
OMNIROUTE_USE_TURBOPACK=0 remains the webpack escape hatch (Windows /
native-binding / bundler-compat issues), and only the documented '0'
opts out — junk values keep the default.
Benchmarked on this codebase (same tree, Next 16.2.9): webpack 1035s vs
Turbopack 539s on a 32-core box; ~20min vs 6min59s on ubuntu-latest.
Artifact validated end-to-end (standalone smoke + e2e/package-artifact/
electron-package-smoke CI jobs, Docker amd64+arm64 builds clean with the
v3.8.27 ImportTracer panic gone on 16.2.9).
TDD: tests/unit/build-bundler-default-turbopack.test.ts (new) +
run-next-playwright.test.ts extended with the unset-env default case;
both red before the flip, green after. ENVIRONMENT.md updated.
Next 16 ships Turbopack as the stable production bundler. Benchmarked on a
32-core box against the same tree: webpack 1035s -> Turbopack 539s (1.92x).
The build script already supports the switch via OMNIROUTE_USE_TURBOPACK=1
(scripts/build/build-next-isolated.mjs) and next.config.mjs already mirrors
the resolveAlias stubs in the turbopack block, so this only flips the CI env.
The webpack .build/next/cache actions/cache step is removed in the same
commit: Turbopack does not use the webpack cache dir (its persistent FS
cache is experimental and NOT enabled), so restoring ~0.5 GB per run would
be pure wasted download. Revert restores webpack + its cache together.
Validation: standalone output smoke-tested (server.js boots, health 200,
dashboard 307); the 428 build warnings are the known benign 'overly broad
file pattern' static-analysis notices for dynamic fs usage (covered at
runtime by outputFileTracingIncludes). Downstream e2e x9, package-artifact
and electron-package-smoke consume this artifact, so a green CI run here
validates the Turbopack artifact end-to-end. nightly-compat and npm-publish
stay on webpack until this PR proves out.
test-unit x8, test-vitest, test-integration x2 and test-security all had
needs: build but never download the next-build artifact — the dependency
only serialized ~20min of Build wall-clock in front of every test run.
Switch them to needs: changes with the same skip condition Build uses
(docs-only PRs and drafts still skip), so the test chain
(test-unit -> test-coverage -> quality-gate -> sonarqube) now runs in
parallel with Build instead of after it.
Jobs that genuinely consume the artifact keep needs: build unchanged:
test-e2e x9, package-artifact, electron-package-smoke.
Expected effect on a full ci.yml run: critical path drops from
build + tests (~30min+) to max(build, tests) — roughly 15-20min saved
per run, no extra runner minutes beyond starting the same jobs earlier.
feat(providers): add Yuanbao (web) cookie-session provider (#6196). TDD-covered; base-reds only (dast-smoke #6228, docs version-drift, executor-kiro anys — #6145 guard fixed on tip via #6270). Integrated into release/v3.8.45.
Realign the #6145 onboarding-href guard to the #6166 helper refactor (buildProviderDetailsHref). Test-only; unblocks the fast-path unit job across the open PR queue. Base-reds only (dast-smoke #6228, docs version-drift, executor-kiro anys). Integrated into release/v3.8.45.
refactor(dashboard): extract tested buildProviderDetailsHref helper for onboarding wizard (#6166). Behavioral #6144 fix already on tip via #6145; this lands the tested-helper hardening. Thanks @KooshaPari. Integrated into release/v3.8.45.
fix(antigravity): strip trailing assistant prefill for Vertex Claude models (#6114). TDD-covered (6/6), merged on TDD strength per owner. Reds are pre-existing base-red drift on release/v3.8.45. Thanks @anki1kr. Integrated into release/v3.8.45.
fix(api): count tool_use/tool_result/thinking blocks in count_tokens estimate (#6221, port from 9router#2337). TDD-covered (6/6); typed the test casts to clear no-new-eslint. Reds are pre-existing base-red drift on release/v3.8.45 (dast-smoke #6228; executor-kiro.test.ts eslint anys; changelog/package.json version drift) — none introduced by this PR. Thanks @luweiCN. Integrated into release/v3.8.45.
docs(architecture): sync stale DB-layer counts (45+/55 → 95+/110+) across REPOSITORY_MAP, db-schema diagram, llm.txt + 42 i18n mirrors (#6167). Docs-only; check:docs-all passes locally on the reconstruction. Reds are pre-existing base-red drift on release/v3.8.45 (dast-smoke #6228; executor-kiro.test.ts eslint anys; changelog/package.json version drift) — none introduced by this PR. Integrated into release/v3.8.45.
