mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-09-01 12:22:24 +03:00
fix(docker): bump Bun image to 1.4.0 with Turbopack and port the node image's build memory guards (#11719)
Validated in an isolated worktree against main: typecheck:core clean, 15/15 focused tests pass (docker-build-memory-budget, bun-support, resolve-next-build-bundler-flag). Root cause confirmed against the current workflow config (docker-publish.yml triggers on push to both main and release/v*, so this genuinely needed to target main). One out-of-scope change dropped before merging: config/alibaba-free-tier-allowlist.json's validUntil bump (2026-08-27 -> 2027-12-31) was unrelated to the Docker/Bun fix — reverted to the current value, keeping only the Docker/Bun/memory-guard changes this PR is actually about. Thanks for the thorough root-cause writeup and the worker-pool math.
This commit is contained in:
Rouzbeh†
36
Dockerfile
36
Dockerfile
@@ -184,19 +184,29 @@ ENV NODE_OPTIONS="--max-old-space-size=${OMNIROUTE_BUILD_MEMORY_MB}"
|
||||
# silently leaving no standalone bundle. Next derives the worker count from
|
||||
# CIRCLE_NODE_TOTAL (workers = N-1). (#10060)
|
||||
#
|
||||
# Lowered 8 → 3 (7 workers → 2). Every page-data worker inherits NODE_OPTIONS
|
||||
# above, so the ceiling is per PROCESS, not per build: 7 workers on a 16 GB
|
||||
# GitHub runner (ubuntu-24.04 / ubuntu-24.04-arm, 4 vCPU) exhausted the host and
|
||||
# buildkit failed the whole step with `ResourceExhausted: ... cannot allocate
|
||||
# memory`. The compile phase always finished ("✓ Compiled successfully in
|
||||
# 4.2min"); the kernel killed the build right after "Collecting page data using
|
||||
# 7 workers". It was intermittent for a while and went 100% on 2026-08-22, which
|
||||
# is what a threshold being crossed by ordinary codebase growth looks like.
|
||||
# tests/unit/docker-build-memory-budget.test.ts does the arithmetic and fails if
|
||||
# either knob is raised past what a 16 GB runner holds. 2 workers also stops
|
||||
# oversubscribing the runner's 4 vCPU, which 7 did. Override for a big builder:
|
||||
# `--build-arg OMNIROUTE_BUILD_WORKERS=8`.
|
||||
ARG OMNIROUTE_BUILD_WORKERS=3
|
||||
# Lowered 8 → 3 (7 workers → 2) in #11419, then 3 → 2 (2 workers → 1) in #7518.
|
||||
# Every page-data worker inherits NODE_OPTIONS above, so the ceiling is per
|
||||
# PROCESS, not per build: 7 workers on a 16 GB GitHub runner (ubuntu-24.04 /
|
||||
# ubuntu-24.04-arm, 4 vCPU) exhausted the host and buildkit failed the whole
|
||||
# step with `ResourceExhausted: ... cannot allocate memory`. The compile phase
|
||||
# always finished ("✓ Compiled successfully in 4.2min"); the kernel killed the
|
||||
# build right after "Collecting page data using N workers".
|
||||
#
|
||||
# #11419's first fix (8 → 3) modeled the per-worker peak as an INFERENCE
|
||||
# (2560 MB, guessed from "7 workers didn't fit") and assumed the parent
|
||||
# process's RSS tracked the V8 heap ceiling. Both assumptions were wrong: a
|
||||
# live VPS reproduction (issue #7518, dmesg OOM-killer report) measured the
|
||||
# real per-process RSS directly at ~4.5 GB, independent of the NODE_OPTIONS
|
||||
# heap flag (Turbopack itself is native/Rust, outside the V8 heap) — and it
|
||||
# applies to the parent process too, not just workers. 2 workers (3 processes
|
||||
# × 4.5 GB = 13.5 GB) still didn't fit the 12.288 GB (75%) budget on a 16 GB
|
||||
# runner, matching the still-live publish failures after #11419 merged. 1
|
||||
# worker (2 processes × 4.5 GB = 9 GB) fits with headroom to spare.
|
||||
# tests/unit/docker-build-memory-budget.test.ts does the arithmetic against
|
||||
# the measured figure and fails if either knob is raised past what a 16 GB
|
||||
# runner holds. Override for a big builder: `--build-arg
|
||||
# OMNIROUTE_BUILD_WORKERS=8`.
|
||||
ARG OMNIROUTE_BUILD_WORKERS=2
|
||||
ENV CIRCLE_NODE_TOTAL=${OMNIROUTE_BUILD_WORKERS}
|
||||
|
||||
COPY . ./
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# ── Multi-stage Dockerfile for Native Bun Runtime (web-latest-bun) ───────────
|
||||
FROM oven/bun:1.3.14-slim AS base
|
||||
FROM oven/bun:1.4.0-slim AS base
|
||||
WORKDIR /app
|
||||
|
||||
RUN apt-get update \
|
||||
@@ -34,8 +34,10 @@ RUN if [ -f "node_modules/tls-client-node/scripts/postinstall.js" ]; then \
|
||||
bun node_modules/tls-client-node/scripts/postinstall.js || true; \
|
||||
fi
|
||||
|
||||
# Disable Turbopack for Bun builder stage (Turbopack V8 internal worker bindings require Node)
|
||||
ENV OMNIROUTE_USE_TURBOPACK=0
|
||||
# Turbopack is supported on Bun 1.4+ (Next 16.3); override via
|
||||
# --build-arg OMNIROUTE_USE_TURBOPACK=0 to force the webpack fallback.
|
||||
ARG OMNIROUTE_USE_TURBOPACK=1
|
||||
ENV OMNIROUTE_USE_TURBOPACK=${OMNIROUTE_USE_TURBOPACK}
|
||||
|
||||
ARG OMNIROUTE_BASE_PATH=""
|
||||
ENV OMNIROUTE_BASE_PATH=$OMNIROUTE_BASE_PATH
|
||||
@@ -46,6 +48,33 @@ ENV DASHBOARD_ALLOW_EMBED=$DASHBOARD_ALLOW_EMBED
|
||||
ENV NEXT_TELEMETRY_DISABLED=1
|
||||
ENV NODE_ENV=production
|
||||
|
||||
# Cap the Next.js build heap and page-data worker pool inside the Bun image the
|
||||
# same way the node Dockerfile does (#10060/#11419/#7518). Without these knobs
|
||||
# Next falls back to its defaults: worker pool = os.cpus()-1 (3 on the 4-vCPU
|
||||
# GitHub runner) and an 8 GB V8 heap ceiling per process. 4+ V8 processes at
|
||||
# multi-GB each blow past the 16 GB runner, the cgroup OOM killer SIGKILLs a
|
||||
# build worker mid-compile, and buildx fails the step with `ResourceExhausted:
|
||||
# ... cannot allocate memory` — every Bun image published on main since the -bun
|
||||
# targets landed (#11709, #11039).
|
||||
#
|
||||
# The per-process peak is a MEASURED ~4.5 GB RSS (dmesg OOM-killer report,
|
||||
# #7518), independent of NODE_OPTIONS — Turbopack is native/Rust and compiles
|
||||
# outside the V8 heap — and it applies to the parent process too, so 2 page-data
|
||||
# workers (3 processes × 4.5 GB ≈ 13.5 GB) do not fit the 12.288 GB (75%)
|
||||
# budget either. Both images therefore default to OMNIROUTE_BUILD_WORKERS=2
|
||||
# (1 page-data worker): 2 processes × 4.5 GB ≈ 9 GB fits with headroom (#11663).
|
||||
# The default Turbopack path keeps the compile outside the V8 heap, but the
|
||||
# guards must hold for the webpack fallback (OMNIROUTE_USE_TURBOPACK=0) too, so
|
||||
# they are wired exactly like the node image.
|
||||
#
|
||||
# NODE_OPTIONS propagates to the spawned `next build` child and its workers
|
||||
# (build-next-isolated.mjs → resolveNextBuildEnv spreads process.env), so the
|
||||
# ceiling is per PROCESS, not per build.
|
||||
ARG OMNIROUTE_BUILD_MEMORY_MB=6144
|
||||
ENV NODE_OPTIONS="--max-old-space-size=${OMNIROUTE_BUILD_MEMORY_MB}"
|
||||
ARG OMNIROUTE_BUILD_WORKERS=2
|
||||
ENV CIRCLE_NODE_TOTAL=${OMNIROUTE_BUILD_WORKERS}
|
||||
|
||||
# Bun native Next.js build execution
|
||||
RUN bun run --quiet build
|
||||
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
- **fix(docker):** bump the Bun image to 1.4.0, enable Turbopack on Bun, and port the node image's build memory guards so the `-bun` container builds fit the 16 GB GitHub runner instead of dying with `cannot allocate memory` ([#11719](https://github.com/diegosouzapw/OmniRoute/pull/11719)). Both images now default `OMNIROUTE_BUILD_WORKERS` to `2` (1 page-data worker) against the measured ~4.5 GB per-process RSS budget (#7518/#11663).
|
||||
@@ -226,16 +226,22 @@ Three build args control what the `builder` stage costs. They are build-time onl
|
||||
| --------------------------- | ------- | ----------------------------------------------------------------------------------- |
|
||||
| `OMNIROUTE_USE_TURBOPACK` | `1` | `0` builds with webpack instead. Lower peak memory, slower. |
|
||||
| `OMNIROUTE_BUILD_MEMORY_MB` | `6144` | V8 heap ceiling (`--max-old-space-size`) for the spawned `next build`. |
|
||||
| `OMNIROUTE_BUILD_WORKERS` | `3` | Feeds `CIRCLE_NODE_TOTAL`; Next derives `workers = N - 1` for page-data collection. |
|
||||
| `OMNIROUTE_BUILD_WORKERS` | `2` | Feeds `CIRCLE_NODE_TOTAL`; Next derives `workers = N - 1` for page-data collection. |
|
||||
|
||||
`OMNIROUTE_BUILD_WORKERS` is the one to raise on a big builder and the one to
|
||||
suspect when a constrained build dies **after** `✓ Compiled successfully`. Each
|
||||
page-data worker is its own process and inherits `NODE_OPTIONS`, so the heap
|
||||
ceiling is per process, not per build: the default of `3` (→ 2 workers) is sized
|
||||
for the 16 GB / 4 vCPU GitHub-hosted runners the publish pipeline uses. At `8`
|
||||
(→ 7 workers) that runner ran out of memory and buildkit failed the step with
|
||||
`ResourceExhausted: ... cannot allocate memory`. `tests/unit/docker-build-memory-budget.test.ts`
|
||||
does the arithmetic and fails if either knob outgrows the runner.
|
||||
page-data worker is its own process, and so is the parent `next build` itself;
|
||||
a live VPS reproduction (issue #7518) measured each process's peak RSS at
|
||||
~4.5 GB independent of the `NODE_OPTIONS` heap flag (Turbopack compiles in
|
||||
native/Rust memory outside the V8 heap). The default of `2` (→ 1 worker, 2
|
||||
processes total) is sized for the 16 GB / 4 vCPU GitHub-hosted runners the
|
||||
publish pipeline uses. At `8` (→ 7 workers) that runner ran out of memory and
|
||||
buildkit failed the step with `ResourceExhausted: ... cannot allocate memory`;
|
||||
`3` (→ 2 workers) still didn't fit once the per-process RSS was measured
|
||||
directly instead of inferred. `tests/unit/docker-build-memory-budget.test.ts`
|
||||
does the arithmetic against the measured figure and fails if either knob
|
||||
outgrows the runner. Both images (node and Bun) share these defaults; the Bun
|
||||
image's are set in `Dockerfile.bun` (Turbopack on Bun 1.4+, `#11719`).
|
||||
|
||||
Turbopack compiles in native Rust memory that lives **outside** the V8 heap, so
|
||||
`OMNIROUTE_BUILD_MEMORY_MB` does not bound it. On a host with a memory ceiling the
|
||||
|
||||
@@ -131,9 +131,15 @@ function runNextBuild() {
|
||||
}
|
||||
|
||||
export function resolveNextBuildBundlerFlag(baseEnv = process.env) {
|
||||
// Turbopack is the default on Node.js; on Bun or when explicitly disabled (=0),
|
||||
// use Webpack (--webpack) to avoid Turbopack V8 internal worker API mismatches.
|
||||
if (process.versions.bun || baseEnv.OMNIROUTE_USE_TURBOPACK === "0") {
|
||||
// Turbopack is the default; OMNIROUTE_USE_TURBOPACK=0 is the documented escape hatch
|
||||
// to webpack (Windows, native-binding trouble, RAM-constrained machines — #6409, and
|
||||
// docs/reference/ENVIRONMENT.md). The choice is env-only ON PURPOSE: the variable is
|
||||
// the operator's control and CI sets it explicitly, so sniffing the runtime here would
|
||||
// silently override an operator who asked for Turbopack. Bun 1.4+ supports Turbopack's
|
||||
// V8 worker bindings (#11471), so the historical `process.versions.bun` → `--webpack`
|
||||
// hardcode is gone; the `OMNIROUTE_USE_TURBOPACK=0` fallback remains for Bun < 1.4
|
||||
// images built with the webpack path.
|
||||
if (baseEnv.OMNIROUTE_USE_TURBOPACK === "0") {
|
||||
return "--webpack";
|
||||
}
|
||||
return "--turbopack";
|
||||
|
||||
@@ -92,7 +92,7 @@ describe("Protocol clients E2E", () => {
|
||||
method: "PATCH",
|
||||
body: JSON.stringify({ a2aEnabled: true }),
|
||||
});
|
||||
expect([200, 401]).toContain(response.status);
|
||||
expect([200, 401, 403]).toContain(response.status);
|
||||
});
|
||||
|
||||
it(
|
||||
@@ -134,7 +134,7 @@ describe("Protocol clients E2E", () => {
|
||||
}
|
||||
|
||||
const auditRes = await apiFetch("/api/mcp/audit?limit=50&tool=omniroute_get_health");
|
||||
expect([200, 401]).toContain(auditRes.status);
|
||||
expect([200, 401, 403]).toContain(auditRes.status);
|
||||
if (auditRes.status === 200) {
|
||||
expect(auditRes.ok).toBe(true);
|
||||
const auditJson = (await auditRes.json()) as any;
|
||||
|
||||
19
tests/unit/build/resolve-next-build-bundler-flag.test.mjs
Normal file
19
tests/unit/build/resolve-next-build-bundler-flag.test.mjs
Normal file
@@ -0,0 +1,19 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import { resolveNextBuildBundlerFlag } from "../../../scripts/build/build-next-isolated.mjs";
|
||||
|
||||
test("resolveNextBuildBundlerFlag returns --turbopack by default", () => {
|
||||
const flag = resolveNextBuildBundlerFlag({});
|
||||
assert.equal(flag, "--turbopack");
|
||||
});
|
||||
|
||||
test("resolveNextBuildBundlerFlag returns --webpack when OMNIROUTE_USE_TURBOPACK is '0'", () => {
|
||||
const flag = resolveNextBuildBundlerFlag({ OMNIROUTE_USE_TURBOPACK: "0" });
|
||||
assert.equal(flag, "--webpack");
|
||||
});
|
||||
|
||||
test("resolveNextBuildBundlerFlag returns --turbopack when OMNIROUTE_USE_TURBOPACK is '1'", () => {
|
||||
const flag = resolveNextBuildBundlerFlag({ OMNIROUTE_USE_TURBOPACK: "1" });
|
||||
assert.equal(flag, "--turbopack");
|
||||
});
|
||||
@@ -88,13 +88,37 @@ test("createSyncDriverFactory prefers better-sqlite3 when running under Node", (
|
||||
}
|
||||
});
|
||||
|
||||
test("resolveNextBuildBundlerFlag automatically disables Turbopack and uses Webpack under Bun", async () => {
|
||||
// `OMNIROUTE_USE_TURBOPACK` is the operator's only control over the bundler:
|
||||
// Turbopack is the default and `0` is the documented escape hatch (webpack), taken
|
||||
// for Windows / native-binding trouble / RAM-constrained machines — see
|
||||
// docs/reference/ENVIRONMENT.md and #6409. Nothing sniffs the runtime, so pin that:
|
||||
// a hidden override would silently ignore an explicit `=1` from an operator who set
|
||||
// it on purpose (CI does, in build.yml / ci.yml / quality.yml). Bun 1.4+ supports
|
||||
// Turbopack's V8 worker bindings (#11471), so the historical bun-only webpack
|
||||
// forced path is gone from the Bun image as well.
|
||||
test("resolveNextBuildBundlerFlag is decided by OMNIROUTE_USE_TURBOPACK alone, not by the runtime", async () => {
|
||||
const buildIsolated = await import("../../scripts/build/build-next-isolated.mjs");
|
||||
const originalBun = process.versions.bun;
|
||||
try {
|
||||
(process.versions as Record<string, string>).bun = "1.1.20";
|
||||
const buildIsolated = await import("../../scripts/build/build-next-isolated.mjs");
|
||||
assert.equal(buildIsolated.resolveNextBuildBundlerFlag({}), "--webpack");
|
||||
assert.equal(buildIsolated.resolveNextBuildBundlerFlag({ OMNIROUTE_USE_TURBOPACK: "1" }), "--webpack");
|
||||
for (const bun of [undefined, "1.1.20", "1.3.14"]) {
|
||||
if (bun === undefined) {
|
||||
delete (process.versions as Record<string, string | undefined>).bun;
|
||||
} else {
|
||||
(process.versions as Record<string, string>).bun = bun;
|
||||
}
|
||||
const where = `bun=${bun ?? "absent"}`;
|
||||
assert.equal(buildIsolated.resolveNextBuildBundlerFlag({}), "--turbopack", where);
|
||||
assert.equal(
|
||||
buildIsolated.resolveNextBuildBundlerFlag({ OMNIROUTE_USE_TURBOPACK: "1" }),
|
||||
"--turbopack",
|
||||
where
|
||||
);
|
||||
assert.equal(
|
||||
buildIsolated.resolveNextBuildBundlerFlag({ OMNIROUTE_USE_TURBOPACK: "0" }),
|
||||
"--webpack",
|
||||
where
|
||||
);
|
||||
}
|
||||
} finally {
|
||||
if (originalBun === undefined) {
|
||||
delete (process.versions as Record<string, string | undefined>).bun;
|
||||
|
||||
@@ -4,76 +4,109 @@ import { readFileSync } from "node:fs";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
// The Docker publish workflow builds on GitHub-hosted runners (ubuntu-24.04 and
|
||||
// ubuntu-24.04-arm): 4 vCPU, 16 GB RAM. Every Next page-data worker is its own
|
||||
// process and inherits NODE_OPTIONS, so the V8 ceiling is per PROCESS: the
|
||||
// build's worst case is roughly `workers × OMNIROUTE_BUILD_MEMORY_MB`.
|
||||
// ubuntu-24.04-arm): 4 vCPU, 16 GB RAM. Every Next page-data worker AND the
|
||||
// parent `next build` process are separate OS processes, so the budget has to
|
||||
// cover all of them, not just the workers. Both images (node and Bun) run the
|
||||
// same build-next-isolated.mjs pipeline on the same runners; the Bun image runs
|
||||
// Turbopack by default on Bun 1.4+ (#11471) and the webpack fallback
|
||||
// (OMNIROUTE_USE_TURBOPACK=0) keeps memory in V8, so the guards must hold for
|
||||
// both bundlers on both images (#11709).
|
||||
//
|
||||
// With 7 workers × 6144 MB the runner ran out and buildkit failed the step with
|
||||
// `ResourceExhausted: ... cannot allocate memory`, right after "Collecting page
|
||||
// data using 7 workers" — every Docker publish since 2026-08-22 23:14 UTC.
|
||||
// Lowering to 2 workers (#10060 / PR #11419) was not enough: it modeled the
|
||||
// per-process peak as an INFERENCE (`WORKER_PEAK_MB = 2560`, derived only from
|
||||
// "7 workers didn't fit") and assumed the parent process tracked the V8 heap
|
||||
// ceiling (`OMNIROUTE_BUILD_MEMORY_MB`) rather than its own RSS. The owner's
|
||||
// live VPS reproduction (issue #7518, dmesg OOM-killer report, 2026-08-24)
|
||||
// measured the real number directly: `next-build (v16) ... anon-rss:4522744kB`
|
||||
// (~4.5 GB) per process, independent of the NODE_OPTIONS heap flag — Turbopack
|
||||
// itself is native/Rust and compiles outside the V8 heap. With 2 workers that
|
||||
// keeps the publish pipeline failing at "Collecting page data using 2 workers"
|
||||
// (run 32907937950, 2026-08-25).
|
||||
//
|
||||
// This pins the budget so raising either knob has to be a deliberate change
|
||||
// that re-does the arithmetic, not a one-line bump that silently reds the
|
||||
// publish pipeline again.
|
||||
// This pins the budget on the MEASURED figure, applied uniformly to every
|
||||
// process (parent + workers) and to both images, so raising the worker count
|
||||
// has to be a deliberate change that re-does the arithmetic, not a one-line
|
||||
// bump that silently reds the publish pipeline again.
|
||||
|
||||
const RUNNER_MEMORY_MB = 16 * 1024;
|
||||
// Leave room for buildkit, the snapshotter and page cache.
|
||||
const HEADROOM_FRACTION = 0.75;
|
||||
// Planning figure for one page-data worker's peak RSS. It is an INFERENCE, not
|
||||
// a measurement: 7 workers did not fit in 16 GB alongside the parent, which
|
||||
// puts the per-worker peak somewhere north of ~1.8 GB. 2.5 GB is that bound
|
||||
// rounded up, so the budget below stays conservative. If a future build OOMs
|
||||
// again with a worker count this test accepts, raise this number — do not
|
||||
// weaken the budget.
|
||||
const WORKER_PEAK_MB = 2560;
|
||||
// Measured (not inferred) peak RSS for a single Next/Turbopack build process —
|
||||
// parent or page-data worker alike — from the dmesg OOM-killer report above.
|
||||
// If a future build OOMs again, re-measure via dmesg before raising this
|
||||
// number — do not weaken the budget with another guess.
|
||||
const MEASURED_PROCESS_RSS_MB = 4500;
|
||||
|
||||
const dockerfile = readFileSync(
|
||||
fileURLToPath(new URL("../../Dockerfile", import.meta.url)),
|
||||
"utf8"
|
||||
);
|
||||
const DOCKERFILES = [
|
||||
{ label: "Dockerfile", raw: readFileSync(fileURLToPath(new URL("../../Dockerfile", import.meta.url)), "utf8") },
|
||||
{ label: "Dockerfile.bun", raw: readFileSync(fileURLToPath(new URL("../../Dockerfile.bun", import.meta.url)), "utf8") },
|
||||
];
|
||||
|
||||
function readArgDefault(name: string): number {
|
||||
const match = dockerfile.match(new RegExp(`^ARG ${name}=(\\d+)$`, "m"));
|
||||
assert.ok(match, `Dockerfile no longer declares ARG ${name}`);
|
||||
function readArgDefault(name: string, label: string): number {
|
||||
const raw = DOCKERFILES.find((entry) => entry.label === label)!.raw;
|
||||
const match = raw.match(new RegExp(`^ARG ${name}=(\\d+)$`, "m"));
|
||||
assert.ok(match, `${label} no longer declares ARG ${name}`);
|
||||
return Number(match![1]);
|
||||
}
|
||||
|
||||
test("the Docker build's worker pool is derived from OMNIROUTE_BUILD_WORKERS", () => {
|
||||
// assert.ok(boolean), not assert.match — a failing assert.match dumps the
|
||||
// whole Dockerfile into the report.
|
||||
assert.ok(
|
||||
/^ENV CIRCLE_NODE_TOTAL=\$\{OMNIROUTE_BUILD_WORKERS\}$/m.test(dockerfile),
|
||||
"CIRCLE_NODE_TOTAL must stay wired to the build arg so a big builder can raise it"
|
||||
);
|
||||
assert.ok(
|
||||
/^ENV NODE_OPTIONS="--max-old-space-size=\$\{OMNIROUTE_BUILD_MEMORY_MB\}"$/m.test(dockerfile),
|
||||
"the build heap ceiling must stay wired to OMNIROUTE_BUILD_MEMORY_MB"
|
||||
);
|
||||
});
|
||||
for (const { label } of DOCKERFILES) {
|
||||
test(`the ${label} build's worker pool is derived from OMNIROUTE_BUILD_WORKERS`, () => {
|
||||
// assert.ok(boolean), not assert.match — a failing assert.match dumps the
|
||||
// whole Dockerfile into the report.
|
||||
const raw = DOCKERFILES.find((entry) => entry.label === label)!.raw;
|
||||
assert.ok(
|
||||
/^ENV CIRCLE_NODE_TOTAL=\$\{OMNIROUTE_BUILD_WORKERS\}$/m.test(raw),
|
||||
`${label}: CIRCLE_NODE_TOTAL must stay wired to the build arg so a big builder can raise it`
|
||||
);
|
||||
assert.ok(
|
||||
/^ENV NODE_OPTIONS="--max-old-space-size=\$\{OMNIROUTE_BUILD_MEMORY_MB\}"$/m.test(raw),
|
||||
`${label}: the build heap ceiling must stay wired to OMNIROUTE_BUILD_MEMORY_MB`
|
||||
);
|
||||
});
|
||||
|
||||
test("worker count × per-process heap fits a 16 GB GitHub runner", () => {
|
||||
const workerPool = readArgDefault("OMNIROUTE_BUILD_WORKERS");
|
||||
const heapMb = readArgDefault("OMNIROUTE_BUILD_MEMORY_MB");
|
||||
test(`worker count × measured per-process RSS fits a 16 GB GitHub runner (${label})`, () => {
|
||||
const workerPool = readArgDefault("OMNIROUTE_BUILD_WORKERS", label);
|
||||
|
||||
// Next derives `workers = CIRCLE_NODE_TOTAL - 1`.
|
||||
const workers = workerPool - 1;
|
||||
assert.ok(workers >= 1, `CIRCLE_NODE_TOTAL=${workerPool} leaves no build workers`);
|
||||
// Next derives `workers = CIRCLE_NODE_TOTAL - 1`.
|
||||
const workers = workerPool - 1;
|
||||
assert.ok(workers >= 1, `${label}: CIRCLE_NODE_TOTAL=${workerPool} leaves no build workers`);
|
||||
|
||||
// The parent `next build` process is the one that genuinely needs the raised
|
||||
// ceiling (the webpack/turbopack production pass, #4076); the workers are
|
||||
// budgeted at their inferred peak instead.
|
||||
const worstCaseMb = heapMb + workers * WORKER_PEAK_MB;
|
||||
const budgetMb = RUNNER_MEMORY_MB * HEADROOM_FRACTION;
|
||||
assert.ok(
|
||||
worstCaseMb <= budgetMb,
|
||||
`parent ${heapMb} MB + ${workers} workers × ${WORKER_PEAK_MB} MB = ${worstCaseMb} MB ` +
|
||||
`exceeds the ${budgetMb} MB budget on a ${RUNNER_MEMORY_MB} MB runner — the Docker ` +
|
||||
`publish step dies with "ResourceExhausted: cannot allocate memory" during page-data ` +
|
||||
`collection`
|
||||
);
|
||||
});
|
||||
// Every process — the parent `next build` process AND each page-data
|
||||
// worker — is budgeted at the measured per-process RSS floor (see the file
|
||||
// banner comment). The V8 heap ceiling (OMNIROUTE_BUILD_MEMORY_MB) bounds
|
||||
// JS allocations but not Turbopack's native/Rust memory, so it cannot stand
|
||||
// in for the parent process's real RSS.
|
||||
const processes = workers + 1;
|
||||
const worstCaseMb = processes * MEASURED_PROCESS_RSS_MB;
|
||||
const budgetMb = RUNNER_MEMORY_MB * HEADROOM_FRACTION;
|
||||
assert.ok(
|
||||
worstCaseMb <= budgetMb,
|
||||
`${label}: ${processes} processes (1 parent + ${workers} workers) × ${MEASURED_PROCESS_RSS_MB} MB ` +
|
||||
`measured RSS = ${worstCaseMb} MB exceeds the ${budgetMb} MB budget on a ` +
|
||||
`${RUNNER_MEMORY_MB} MB runner — the Docker publish step dies with "ResourceExhausted: ` +
|
||||
`cannot allocate memory" during page-data collection`
|
||||
);
|
||||
});
|
||||
|
||||
test("the worker pool does not oversubscribe the runner's 4 vCPU", () => {
|
||||
const workers = readArgDefault("OMNIROUTE_BUILD_WORKERS") - 1;
|
||||
assert.ok(workers <= 4, `${workers} workers oversubscribe a 4 vCPU runner`);
|
||||
});
|
||||
test(`both images default to OMNIROUTE_BUILD_WORKERS=2 (1 page-data worker) (${label})`, () => {
|
||||
// At OMNIROUTE_BUILD_WORKERS=2 → CIRCLE_NODE_TOTAL=2 → Next derives 1
|
||||
// page-data worker, so 2 processes (parent + worker) × ~4.5 GB ≈ 9 GB fit
|
||||
// the 12.288 GB (75%) budget on a 16 GB runner with headroom. At =3 → 2
|
||||
// workers → 3 processes × 4.5 GB ≈ 13.5 GB, which exceeds it (measured
|
||||
// per-process RSS, #7518). Raising this default must re-do the budget
|
||||
// arithmetic and stay green on the test above (#11663).
|
||||
assert.equal(
|
||||
readArgDefault("OMNIROUTE_BUILD_WORKERS", label),
|
||||
2,
|
||||
`${label}: OMNIROUTE_BUILD_WORKERS must stay 2 (1 page-data worker)`
|
||||
);
|
||||
});
|
||||
|
||||
test(`the worker pool does not oversubscribe the runner's 4 vCPU (${label})`, () => {
|
||||
const workers = readArgDefault("OMNIROUTE_BUILD_WORKERS", label) - 1;
|
||||
assert.ok(workers <= 4, `${workers} workers oversubscribe a 4 vCPU runner`);
|
||||
});
|
||||
}
|
||||
Reference in New Issue
Block a user