mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-25 16:42:16 +03:00
8bbe92c692
Every "Publish to Docker Hub" run has failed since 2026-08-22 23:14 UTC — 96 of
the last 100. The builder stage dies with:
ERROR: failed to solve: ResourceExhausted: process "/bin/sh -c ... npm run
build ..." did not complete successfully: cannot allocate memory
That is the kernel, not V8. The log puts it precisely: the compile phase always
finishes ("✓ Compiled successfully in 4.2min") and the build is killed right
after "Collecting page data using 7 workers".
Each page-data worker is its own process and inherits NODE_OPTIONS, so the
--max-old-space-size ceiling is per PROCESS, not per build. CIRCLE_NODE_TOTAL=8
means 7 workers, and 7 of them alongside the parent no longer fit the 16 GB /
4 vCPU GitHub-hosted runners the pipeline builds on. It was intermittent for a
while before going 100%, which is what a threshold crossed by ordinary codebase
growth looks like — 7 was also oversubscribing a 4 vCPU runner.
Lower the pool to 3 (2 workers) and make it a build arg, so a big builder can
raise it back with `--build-arg OMNIROUTE_BUILD_WORKERS=8`.
tests/unit/docker-build-memory-budget.test.ts pins the budget: it reads the two
ARG defaults out of the Dockerfile and fails if `parent heap + workers × peak`
outgrows the runner, or if the pool oversubscribes its CPUs. Red on the base
(3/3), green here (3/3). The per-worker peak it budgets with is documented as an
inference from this failure, not a measurement.
DOCKER_GUIDE's build-arg table was stale (it still listed the pre-#10060 4096 MB
default); updated and given the new knob plus the symptom to recognize.
CIRCLE_NODE_TOTAL and OMNIROUTE_BUILD_WORKERS are allowlisted in the
fabricated-docs gate with the reason: neither is read via process.env here — one
is a Dockerfile ARG, the other is read by Next itself.
Note: the real proof is the next publish run. This failure mode only reproduces
on a memory-constrained host, so it cannot be reproduced by the unit suite; the
test guards the arithmetic, not the outcome.
Co-authored-by: Xiangzhe <bakryun0718@proton.me>
80 lines
3.6 KiB
TypeScript
80 lines
3.6 KiB
TypeScript
import test from "node:test";
|
||
import assert from "node:assert/strict";
|
||
import { readFileSync } from "node:fs";
|
||
import { fileURLToPath } from "node:url";
|
||
|
||
// The Docker publish workflow builds on GitHub-hosted runners (ubuntu-24.04 and
|
||
// ubuntu-24.04-arm): 4 vCPU, 16 GB RAM. Every Next page-data worker is its own
|
||
// process and inherits NODE_OPTIONS, so the V8 ceiling is per PROCESS: the
|
||
// build's worst case is roughly `workers × OMNIROUTE_BUILD_MEMORY_MB`.
|
||
//
|
||
// With 7 workers × 6144 MB the runner ran out and buildkit failed the step with
|
||
// `ResourceExhausted: ... cannot allocate memory`, right after "Collecting page
|
||
// data using 7 workers" — every Docker publish since 2026-08-22 23:14 UTC.
|
||
//
|
||
// This pins the budget so raising either knob has to be a deliberate change
|
||
// that re-does the arithmetic, not a one-line bump that silently reds the
|
||
// publish pipeline again.
|
||
|
||
const RUNNER_MEMORY_MB = 16 * 1024;
|
||
// Leave room for buildkit, the snapshotter and page cache.
|
||
const HEADROOM_FRACTION = 0.75;
|
||
// Planning figure for one page-data worker's peak RSS. It is an INFERENCE, not
|
||
// a measurement: 7 workers did not fit in 16 GB alongside the parent, which
|
||
// puts the per-worker peak somewhere north of ~1.8 GB. 2.5 GB is that bound
|
||
// rounded up, so the budget below stays conservative. If a future build OOMs
|
||
// again with a worker count this test accepts, raise this number — do not
|
||
// weaken the budget.
|
||
const WORKER_PEAK_MB = 2560;
|
||
|
||
const dockerfile = readFileSync(
|
||
fileURLToPath(new URL("../../Dockerfile", import.meta.url)),
|
||
"utf8"
|
||
);
|
||
|
||
function readArgDefault(name: string): number {
|
||
const match = dockerfile.match(new RegExp(`^ARG ${name}=(\\d+)$`, "m"));
|
||
assert.ok(match, `Dockerfile no longer declares ARG ${name}`);
|
||
return Number(match![1]);
|
||
}
|
||
|
||
test("the Docker build's worker pool is derived from OMNIROUTE_BUILD_WORKERS", () => {
|
||
// assert.ok(boolean), not assert.match — a failing assert.match dumps the
|
||
// whole Dockerfile into the report.
|
||
assert.ok(
|
||
/^ENV CIRCLE_NODE_TOTAL=\$\{OMNIROUTE_BUILD_WORKERS\}$/m.test(dockerfile),
|
||
"CIRCLE_NODE_TOTAL must stay wired to the build arg so a big builder can raise it"
|
||
);
|
||
assert.ok(
|
||
/^ENV NODE_OPTIONS="--max-old-space-size=\$\{OMNIROUTE_BUILD_MEMORY_MB\}"$/m.test(dockerfile),
|
||
"the build heap ceiling must stay wired to OMNIROUTE_BUILD_MEMORY_MB"
|
||
);
|
||
});
|
||
|
||
test("worker count × per-process heap fits a 16 GB GitHub runner", () => {
|
||
const workerPool = readArgDefault("OMNIROUTE_BUILD_WORKERS");
|
||
const heapMb = readArgDefault("OMNIROUTE_BUILD_MEMORY_MB");
|
||
|
||
// Next derives `workers = CIRCLE_NODE_TOTAL - 1`.
|
||
const workers = workerPool - 1;
|
||
assert.ok(workers >= 1, `CIRCLE_NODE_TOTAL=${workerPool} leaves no build workers`);
|
||
|
||
// The parent `next build` process is the one that genuinely needs the raised
|
||
// ceiling (the webpack/turbopack production pass, #4076); the workers are
|
||
// budgeted at their inferred peak instead.
|
||
const worstCaseMb = heapMb + workers * WORKER_PEAK_MB;
|
||
const budgetMb = RUNNER_MEMORY_MB * HEADROOM_FRACTION;
|
||
assert.ok(
|
||
worstCaseMb <= budgetMb,
|
||
`parent ${heapMb} MB + ${workers} workers × ${WORKER_PEAK_MB} MB = ${worstCaseMb} MB ` +
|
||
`exceeds the ${budgetMb} MB budget on a ${RUNNER_MEMORY_MB} MB runner — the Docker ` +
|
||
`publish step dies with "ResourceExhausted: cannot allocate memory" during page-data ` +
|
||
`collection`
|
||
);
|
||
});
|
||
|
||
test("the worker pool does not oversubscribe the runner's 4 vCPU", () => {
|
||
const workers = readArgDefault("OMNIROUTE_BUILD_WORKERS") - 1;
|
||
assert.ok(workers <= 4, `${workers} workers oversubscribe a 4 vCPU runner`);
|
||
});
|