* fix(ci): stop hosted docker-publish OOM and unpaint Build (advisory)
docker-publish was firing 8 concurrent hosted builds on every merge
storm; each died ResourceExhausted in npm run build (#11976). One
publish per ref, webpack instead of Turbopack so native RSS stays
inside the V8 heap we can cap. Build (advisory) is skipped: continue-on-error
still reports FAILURE and was painting every fork PR red.
Closes#11976
* fix(ci): run docker-publish amd64 on omni-build and share the heavy lane
The .113 box is 31 GB / 32 cores — enough for one next-build. Hosted
ubuntu-24.04 is ~7 GB and ResourceExhausted every publish (#11976).
amd64 now targets [self-hosted, omni-build] (Turbopack) when
USE_VPS_RUNNER is on, joins the existing heavy-build-main group so it
queues beside ci.yml Build instead of becoming a third heavy, and
falls back to hosted + webpack if the VPS is off. arm64 stays on
ubuntu-24.04-arm with webpack (no ARM box).
* test(ci): align the advisory-build contract with the hosted-OOM skip
if: ${{ false }} tripped zizmor obfuscation (194→195). Bare if: false
skips the job without a new finding. The #7307 test now pins the skip
and keeps the job body as the restore recipe.
* test(infra): retry recursive temp-dir removal instead of failing a shard on ENOTEMPTY (#11966)
Two shards on release/v3.8.51 went red in one day with the same signature —
"ENOTEMPTY, Directory not empty: /tmp/omniroute-<test>-XXXXXX" — from
combo-same-provider-cascade (Unit Tests fast-path 4/4, on a PR that touches only
.github/) and auth-policy-embeddings-webfetch-7785 (the 20k-test TIA step). Both pass
alone and on re-run: the cleanup races something still writing into the directory
(SQLite WAL/-shm checkpoint, a worker, the backup) and under a loaded hosted runner
the window opens. 1154 test files do their own cleanup with
fs.rmSync(dir, { recursive: true, force: true }); 57 already asked for retries.
One-shot codemod (scripts/ad-hoc/codemod-rm-maxretries.mjs, kept for the record):
every rm / rmSync / rmdirSync option object with `recursive: true` and no
`maxRetries` gains `maxRetries: 5, retryDelay: 100` — Node itself then retries
ENOTEMPTY/EBUSY/EPERM for up to ~0.5 s before giving up. 2243 call sites in 1292
files under tests/, the shared tests/_setup/isolateDataDir.ts exit hook included.
Only the option object changes: no call site, assertion or import is touched.
Validation: prettier and ESLint (with the frozen suppressions) clean on all 1292
files; a random 20-file sample runs green (quota-redis-store hangs identically on
the untouched tree — it needs a Redis on localhost, an environment matter). The
four unit shards on this PR are the full run.
* fix(quality): let check-forgotten-sibling-tests read a 1,000-file diff
The gate shells out to `git diff` through execFileSync with Node's default 1 MB
maxBuffer; the 1,292-file codemod in this PR is the first diff large enough to
overflow it, and the gate died with `spawnSync git ENOBUFS` before comparing
anything. 64 MB is far above any real PR and costs nothing when unused.
* fix(ci): pin Build (advisory) to a hosted runner with memory provisioning
`Build (advisory)` has been reporting a permanent red on every PR while
producing no usable signal at all.
Measured over the last 25 quality.yml runs (2026-08-14): not one instance of
the job reached a conclusion. Every sample was either queued on the
self-hosted pool — 2 runners, omniroute-113-6/7, both permanently busy; one
job sat queued for over 2 hours and was still unclaimed — or, when it did land
on a runner, killed mid-build by this workflow's own cancel-in-progress
concurrency. All 6 sampled "failures" are exit 143 / "The runner has received
a shutdown signal" at ~3.5 min into `npm run build`. Zero OOM, zero build
errors. The job was consuming a runner the real gates compete for while
telling every PR author it was broken.
Gap 19 deliberately left USE_VPS_RUNNER governing build-like jobs, on the
premise that the build needs the .113's RAM. That premise no longer holds:
`Fast Production Build` (build.yml) runs `build:release` — a superset of this
job's `npm run build`, plus the CLI bundle — on plain ubuntu-latest and passed
24 of its last 25 runs in ~15 min. The difference is memory PROVISIONING, not
the machine: a 10 GB swapfile plus a 12 GB V8 heap. Swap is the part that
matters, because --max-old-space-size bounds only V8's JS heap and never
Turbopack's native Rust allocation (#6409).
Pins the job to ubuntu-latest and mirrors both settings from build.yml.
USE_VPS_RUNNER keeps its other consumers (ci.yml Build, nightly-release-green,
npm-publish), so the variable stays meaningful. Fork safety is strictly
improved: no PR can reach the LAN runner through this job any more.
check:workflows --ratchet: 186 zizmor findings, baseline 190, no regression.
prettier + YAML parse: clean.
* fix(ci): scope Build (advisory) to fork PRs
Follow-up to the hosted-runner pin in this same PR, after measuring what the
job is actually for.
build.yml's `Fast Production Build` triggers on `push: branches: ["**"]` and
runs `build:release` — a superset of this job's `npm run build`, plus the CLI
bundle. For an own-origin branch that push fires here, so the tree was being
built twice per PR. A fork contributor pushes to THEIR repo, so build.yml
never runs in this repo and this job is their only pre-merge build signal.
That could have argued for deleting the job, except the traffic says
otherwise: 72 of the last 100 PRs into release/** come from forks. The fork
case is the majority, not the exception. So the job earns its place — it just
should not duplicate build.yml for the own-origin 28%. Added the fork filter
to the existing `if`.
Also corrects the reliability claim in the previous commit message. Over a
wider window the job is not literally never-green: across 2026-08-13/14 it
reached `success` on roughly 10-15% of runs (13/138 on 08-14, 7/53 sampled on
08-13). Chronically unreliable, not permanently dead — the conclusion and the
fix are unchanged.
The #7307 guard in tests/unit/build/check-workflows.test.ts pinned the old
self-hosted expression, so it is realigned here: it now asserts the hosted
pin, the absence of self-hosted/USE_VPS_RUNNER in the job's DIRECTIVES (the
comment legitimately explains why the pool was abandoned, so the scan strips
comments), both memory settings, and the fork filter. Mutation-validated —
restoring self-hosted, dropping the swapfile, or flipping the fork filter each
turns it red.
check-workflows.test.ts: 32 pass, 0 fail.
check:workflows --ratchet: 186 findings, baseline 190, no regression.
---------
Co-authored-by: Xiangzhe <bakryun0718@proton.me>
Release v3.8.24 — see CHANGELOG.md [3.8.24] for the full notes and the PR description for the contributors hall. Integration of release/v3.8.24 into main.
