mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-27 01:22:10 +03:00
Compare commits
8 Commits
chore/file
...
fix/7518-d
| Author | SHA1 | Date | |
|---|---|---|---|
|
15a786b56f | ||
|
|
91aeca0440 | ||
|
|
49749eb0d0 | ||
|
|
8d17110082 | ||
|
|
1ee4818224 | ||
|
|
6afc7ddab0 | ||
|
|
71fd806bb0 | ||
|
|
adedc351d7 |
36
Dockerfile
36
Dockerfile
@@ -184,19 +184,29 @@ ENV NODE_OPTIONS="--max-old-space-size=${OMNIROUTE_BUILD_MEMORY_MB}"
|
||||
# silently leaving no standalone bundle. Next derives the worker count from
|
||||
# CIRCLE_NODE_TOTAL (workers = N-1). (#10060)
|
||||
#
|
||||
# Lowered 8 → 3 (7 workers → 2). Every page-data worker inherits NODE_OPTIONS
|
||||
# above, so the ceiling is per PROCESS, not per build: 7 workers on a 16 GB
|
||||
# GitHub runner (ubuntu-24.04 / ubuntu-24.04-arm, 4 vCPU) exhausted the host and
|
||||
# buildkit failed the whole step with `ResourceExhausted: ... cannot allocate
|
||||
# memory`. The compile phase always finished ("✓ Compiled successfully in
|
||||
# 4.2min"); the kernel killed the build right after "Collecting page data using
|
||||
# 7 workers". It was intermittent for a while and went 100% on 2026-08-22, which
|
||||
# is what a threshold being crossed by ordinary codebase growth looks like.
|
||||
# tests/unit/docker-build-memory-budget.test.ts does the arithmetic and fails if
|
||||
# either knob is raised past what a 16 GB runner holds. 2 workers also stops
|
||||
# oversubscribing the runner's 4 vCPU, which 7 did. Override for a big builder:
|
||||
# `--build-arg OMNIROUTE_BUILD_WORKERS=8`.
|
||||
ARG OMNIROUTE_BUILD_WORKERS=3
|
||||
# Lowered 8 → 3 (7 workers → 2) in #11419, then 3 → 2 (2 workers → 1) in #7518.
|
||||
# Every page-data worker inherits NODE_OPTIONS above, so the ceiling is per
|
||||
# PROCESS, not per build: 7 workers on a 16 GB GitHub runner (ubuntu-24.04 /
|
||||
# ubuntu-24.04-arm, 4 vCPU) exhausted the host and buildkit failed the whole
|
||||
# step with `ResourceExhausted: ... cannot allocate memory`. The compile phase
|
||||
# always finished ("✓ Compiled successfully in 4.2min"); the kernel killed the
|
||||
# build right after "Collecting page data using N workers".
|
||||
#
|
||||
# #11419's first fix (8 → 3) modeled the per-worker peak as an INFERENCE
|
||||
# (2560 MB, guessed from "7 workers didn't fit") and assumed the parent
|
||||
# process's RSS tracked the V8 heap ceiling. Both assumptions were wrong: a
|
||||
# live VPS reproduction (issue #7518, dmesg OOM-killer report) measured the
|
||||
# real per-process RSS directly at ~4.5 GB, independent of the NODE_OPTIONS
|
||||
# heap flag (Turbopack itself is native/Rust, outside the V8 heap) — and it
|
||||
# applies to the parent process too, not just workers. 2 workers (3 processes
|
||||
# × 4.5 GB = 13.5 GB) still didn't fit the 12.288 GB (75%) budget on a 16 GB
|
||||
# runner, matching the still-live publish failures after #11419 merged. 1
|
||||
# worker (2 processes × 4.5 GB = 9 GB) fits with headroom to spare.
|
||||
# tests/unit/docker-build-memory-budget.test.ts does the arithmetic against
|
||||
# the measured figure and fails if either knob is raised past what a 16 GB
|
||||
# runner holds. Override for a big builder: `--build-arg
|
||||
# OMNIROUTE_BUILD_WORKERS=8`.
|
||||
ARG OMNIROUTE_BUILD_WORKERS=2
|
||||
ENV CIRCLE_NODE_TOTAL=${OMNIROUTE_BUILD_WORKERS}
|
||||
|
||||
COPY . ./
|
||||
|
||||
1
changelog.d/fixes/7518-docker-build-memory-budget.md
Normal file
1
changelog.d/fixes/7518-docker-build-memory-budget.md
Normal file
@@ -0,0 +1 @@
|
||||
- **fix(docker):** re-derive the Docker build's worker-pool memory budget from the MEASURED ~4.5 GB per-process RSS (the issue owner's own VPS dmesg OOM-killer reproduction) instead of the stale 2560 MB/worker inference #11419 shipped, and lower `OMNIROUTE_BUILD_WORKERS` 3 → 2 so 1 parent + 1 page-data worker (2 processes × 4.5 GB = 9 GB) fits the 12.288 GB (75%) budget on a 16 GB GitHub Actions runner — the previous default (1 parent + 2 workers = 13.5 GB) still overcommitted the runner and kept "Publish to Docker Hub" failing with `cannot allocate memory` after #11419 merged (#7518).
|
||||
@@ -226,16 +226,21 @@ Three build args control what the `builder` stage costs. They are build-time onl
|
||||
| --------------------------- | ------- | ----------------------------------------------------------------------------------- |
|
||||
| `OMNIROUTE_USE_TURBOPACK` | `1` | `0` builds with webpack instead. Lower peak memory, slower. |
|
||||
| `OMNIROUTE_BUILD_MEMORY_MB` | `6144` | V8 heap ceiling (`--max-old-space-size`) for the spawned `next build`. |
|
||||
| `OMNIROUTE_BUILD_WORKERS` | `3` | Feeds `CIRCLE_NODE_TOTAL`; Next derives `workers = N - 1` for page-data collection. |
|
||||
| `OMNIROUTE_BUILD_WORKERS` | `2` | Feeds `CIRCLE_NODE_TOTAL`; Next derives `workers = N - 1` for page-data collection. |
|
||||
|
||||
`OMNIROUTE_BUILD_WORKERS` is the one to raise on a big builder and the one to
|
||||
suspect when a constrained build dies **after** `✓ Compiled successfully`. Each
|
||||
page-data worker is its own process and inherits `NODE_OPTIONS`, so the heap
|
||||
ceiling is per process, not per build: the default of `3` (→ 2 workers) is sized
|
||||
for the 16 GB / 4 vCPU GitHub-hosted runners the publish pipeline uses. At `8`
|
||||
(→ 7 workers) that runner ran out of memory and buildkit failed the step with
|
||||
`ResourceExhausted: ... cannot allocate memory`. `tests/unit/docker-build-memory-budget.test.ts`
|
||||
does the arithmetic and fails if either knob outgrows the runner.
|
||||
page-data worker is its own process, and so is the parent `next build` itself;
|
||||
a live VPS reproduction (issue #7518) measured each process's peak RSS at
|
||||
~4.5 GB independent of the `NODE_OPTIONS` heap flag (Turbopack compiles in
|
||||
native/Rust memory outside the V8 heap). The default of `2` (→ 1 worker, 2
|
||||
processes total) is sized for the 16 GB / 4 vCPU GitHub-hosted runners the
|
||||
publish pipeline uses. At `8` (→ 7 workers) that runner ran out of memory and
|
||||
buildkit failed the step with `ResourceExhausted: ... cannot allocate memory`;
|
||||
`3` (→ 2 workers) still didn't fit once the per-process RSS was measured
|
||||
directly instead of inferred. `tests/unit/docker-build-memory-budget.test.ts`
|
||||
does the arithmetic against the measured figure and fails if either knob
|
||||
outgrows the runner.
|
||||
|
||||
Turbopack compiles in native Rust memory that lives **outside** the V8 heap, so
|
||||
`OMNIROUTE_BUILD_MEMORY_MB` does not bound it. On a host with a memory ceiling the
|
||||
|
||||
@@ -90,13 +90,19 @@ async function resolveZaiBrowserAttachments(
|
||||
> {
|
||||
try {
|
||||
// Browser-page upload: keep the original bytes/mimeType (no Cursor wire prep).
|
||||
// EncodedImage.mimeType is optional on the wire type, but every producer
|
||||
// reachable here (decodeDataUrl / fetchImageBytes) validates an image/*
|
||||
// string before pushing; the fallback only satisfies the attachment type.
|
||||
const images = await resolveCursorImages(imageUrls, { prepareForWire: false });
|
||||
return {
|
||||
attachments: images.map((image, index) => ({
|
||||
name: zaiImageFileName(image.mimeType, index),
|
||||
mimeType: image.mimeType,
|
||||
buffer: image.data,
|
||||
})),
|
||||
attachments: images.map((image, index) => {
|
||||
const mimeType = image.mimeType ?? "image/jpeg";
|
||||
return {
|
||||
name: zaiImageFileName(mimeType, index),
|
||||
mimeType,
|
||||
buffer: image.data,
|
||||
};
|
||||
}),
|
||||
};
|
||||
} catch (error) {
|
||||
const message =
|
||||
|
||||
@@ -15,6 +15,7 @@ interface ErrorResponseBody {
|
||||
message: string;
|
||||
type?: string;
|
||||
code?: string;
|
||||
reason?: string;
|
||||
};
|
||||
upstream_details?: Record<string, unknown> | null; // sanitized upstream provider body
|
||||
}
|
||||
@@ -108,6 +109,7 @@ export function sanitizeUpstreamDetails(value: unknown, depth = 0): unknown {
|
||||
export type ErrorBodyClassification = {
|
||||
type?: string;
|
||||
code?: string;
|
||||
reason?: string;
|
||||
};
|
||||
|
||||
/**
|
||||
@@ -132,6 +134,7 @@ export function buildErrorBody(
|
||||
message: safeMessage,
|
||||
type: classification?.type ?? errorInfo.type,
|
||||
code: classification?.code ?? errorInfo.code,
|
||||
reason: classification?.reason,
|
||||
},
|
||||
};
|
||||
|
||||
|
||||
121
package-lock.json
generated
121
package-lock.json
generated
@@ -86,7 +86,6 @@
|
||||
"undici": "^8.10.0",
|
||||
"update-notifier": "^7.3.1",
|
||||
"uuid": "^14.0.0",
|
||||
"wreq-js": "3.1.0",
|
||||
"ws": "^8.21.3",
|
||||
"xxhash-wasm": "^1.1.0",
|
||||
"yazl": "^3.3.1",
|
||||
@@ -110,7 +109,7 @@
|
||||
"@testing-library/react": "^16.3.2",
|
||||
"@testing-library/user-event": "^14.6.6",
|
||||
"@types/better-sqlite3": "^9.6.0",
|
||||
"@types/bun": "*",
|
||||
"@types/bun": "latest",
|
||||
"@types/node": "^26.2.0",
|
||||
"@types/react": "^19.2.18",
|
||||
"@types/react-dom": "^19.2.4",
|
||||
@@ -161,7 +160,7 @@
|
||||
"better-sqlite3": "^13.0.2",
|
||||
"js-tiktoken": "^1.0.20",
|
||||
"keytar": "^7.9.0",
|
||||
"onnxruntime-node": "1.27.0",
|
||||
"onnxruntime-node": "1.24.3",
|
||||
"sqlite-vec": "^0.1.9",
|
||||
"tls-client-node": "^0.2.0",
|
||||
"wreq-js": "^3.1.0"
|
||||
@@ -15121,6 +15120,14 @@
|
||||
"url": "https://opencollective.com/express"
|
||||
}
|
||||
},
|
||||
"node_modules/boolean": {
|
||||
"version": "3.2.0",
|
||||
"resolved": "https://registry.npmjs.org/boolean/-/boolean-3.2.0.tgz",
|
||||
"integrity": "sha512-d0II/GO9uf9lfUHH2BQsjxzRJZBdsjgsBiW4BvhWk/3qoKwQFjIDVN19PfX8F2D/r9PCMTtLWjYVCFrpeYUzsw==",
|
||||
"deprecated": "Package no longer supported. Contact Support at https://www.npmjs.com/support for more info.",
|
||||
"license": "MIT",
|
||||
"optional": true
|
||||
},
|
||||
"node_modules/bottleneck": {
|
||||
"version": "2.19.5",
|
||||
"resolved": "https://registry.npmjs.org/bottleneck/-/bottleneck-2.19.5.tgz",
|
||||
@@ -18206,6 +18213,13 @@
|
||||
"node": ">=8"
|
||||
}
|
||||
},
|
||||
"node_modules/detect-node": {
|
||||
"version": "2.1.0",
|
||||
"resolved": "https://registry.npmjs.org/detect-node/-/detect-node-2.1.0.tgz",
|
||||
"integrity": "sha512-T0NIuQpnTvFDATNuHN5roPwSBG83rFsuO+MXXH9/3N1eFbn4wcPjttvjMLEPWJ0RGUYgQE7cGgS3tNxbqCGM7g==",
|
||||
"license": "MIT",
|
||||
"optional": true
|
||||
},
|
||||
"node_modules/detect-node-es": {
|
||||
"version": "1.1.0",
|
||||
"resolved": "https://registry.npmjs.org/detect-node-es/-/detect-node-es-1.1.0.tgz",
|
||||
@@ -19051,6 +19065,13 @@
|
||||
"benchmarks"
|
||||
]
|
||||
},
|
||||
"node_modules/es6-error": {
|
||||
"version": "4.1.1",
|
||||
"resolved": "https://registry.npmjs.org/es6-error/-/es6-error-4.1.1.tgz",
|
||||
"integrity": "sha512-Um/+FxMr9CISWh0bi5Zv0iOD+4cFh5qLeks1qhAopKVAJw3drgKbKySikp7wGhDL0HPeaja0P5ULZrxLkniUVg==",
|
||||
"license": "MIT",
|
||||
"optional": true
|
||||
},
|
||||
"node_modules/es6-promisify": {
|
||||
"version": "7.0.0",
|
||||
"resolved": "https://registry.npmjs.org/es6-promisify/-/es6-promisify-7.0.0.tgz",
|
||||
@@ -21857,16 +21878,18 @@
|
||||
}
|
||||
},
|
||||
"node_modules/global-agent": {
|
||||
"version": "4.1.3",
|
||||
"resolved": "https://registry.npmjs.org/global-agent/-/global-agent-4.1.3.tgz",
|
||||
"integrity": "sha512-KUJEViiuFT3I97t+GYMikLPJS2Lfo/S2F+DQuBWzuzaMPnvt5yyZePzArx36fBzpGTxZjIpDbXLeySLgh+k76g==",
|
||||
"version": "3.0.0",
|
||||
"resolved": "https://registry.npmjs.org/global-agent/-/global-agent-3.0.0.tgz",
|
||||
"integrity": "sha512-PT6XReJ+D07JvGoxQMkT6qji/jVNfX/h364XHZOWeRzy64sSFr+xJ5OX7LI3b4MPQzdL4H8Y8M0xzPpsVMwA8Q==",
|
||||
"license": "BSD-3-Clause",
|
||||
"optional": true,
|
||||
"dependencies": {
|
||||
"globalthis": "^1.0.2",
|
||||
"matcher": "^4.0.0",
|
||||
"semver": "^7.3.5",
|
||||
"serialize-error": "^8.1.0"
|
||||
"boolean": "^3.0.1",
|
||||
"es6-error": "^4.1.1",
|
||||
"matcher": "^3.0.0",
|
||||
"roarr": "^2.15.3",
|
||||
"semver": "^7.3.2",
|
||||
"serialize-error": "^7.0.1"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=10.0"
|
||||
@@ -25424,6 +25447,13 @@
|
||||
"url": "https://github.com/sponsors/isaacs"
|
||||
}
|
||||
},
|
||||
"node_modules/json-stringify-safe": {
|
||||
"version": "5.0.1",
|
||||
"resolved": "https://registry.npmjs.org/json-stringify-safe/-/json-stringify-safe-5.0.1.tgz",
|
||||
"integrity": "sha512-ZClg6AaYvamvYEE82d3Iyd3vSSIjQ+odgjaTzRuO3s7toCdFKczob2i0zCh7JE8kWn17yvAWhUVxvqGwUalsRA==",
|
||||
"license": "ISC",
|
||||
"optional": true
|
||||
},
|
||||
"node_modules/json5": {
|
||||
"version": "2.2.3",
|
||||
"resolved": "https://registry.npmjs.org/json5/-/json5-2.2.3.tgz",
|
||||
@@ -27292,9 +27322,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/matcher": {
|
||||
"version": "4.0.0",
|
||||
"resolved": "https://registry.npmjs.org/matcher/-/matcher-4.0.0.tgz",
|
||||
"integrity": "sha512-S6x5wmcDmsDRRU/c2dkccDwQPXoFczc5+HpQ2lON8pnvHlnvHAHj5WlLVvw6n6vNyHuVugYrFohYxbS+pvFpKQ==",
|
||||
"version": "3.0.0",
|
||||
"resolved": "https://registry.npmjs.org/matcher/-/matcher-3.0.0.tgz",
|
||||
"integrity": "sha512-OkeDaAZ/bQCxeFAozM55PKcKU0yJMPGifLwV4Qgjitu+5MoAfSQN4lsLJeXZ1b8w0x+/Emda6MZgXS1jvsapng==",
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"dependencies": {
|
||||
@@ -27302,9 +27332,6 @@
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=10"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/sindresorhus"
|
||||
}
|
||||
},
|
||||
"node_modules/material-symbols": {
|
||||
@@ -30277,16 +30304,16 @@
|
||||
}
|
||||
},
|
||||
"node_modules/onnxruntime-common": {
|
||||
"version": "1.27.0",
|
||||
"resolved": "https://registry.npmjs.org/onnxruntime-common/-/onnxruntime-common-1.27.0.tgz",
|
||||
"integrity": "sha512-3KxL5wIVqa8Ex08jxSzncm9CMgw8CjOFyOQ7SxvG9o0cVLlhTNKXyIQuTbtX4tGPJEf73OER2xrjt4HJSBL4ow==",
|
||||
"version": "1.24.3",
|
||||
"resolved": "https://registry.npmjs.org/onnxruntime-common/-/onnxruntime-common-1.24.3.tgz",
|
||||
"integrity": "sha512-GeuPZO6U/LBJXvwdaqHbuUmoXiEdeCjWi/EG7Y1HNnDwJYuk6WUbNXpF6luSUY8yASul3cmUlLGrCCL1ZgVXqA==",
|
||||
"license": "MIT",
|
||||
"optional": true
|
||||
},
|
||||
"node_modules/onnxruntime-node": {
|
||||
"version": "1.27.0",
|
||||
"resolved": "https://registry.npmjs.org/onnxruntime-node/-/onnxruntime-node-1.27.0.tgz",
|
||||
"integrity": "sha512-QEzGwrvNBgv4uPVdnbHsOGG4G6T96mdlcFI8aAKPjMU8wOPpVocPXb6k3QGkaZagVTv2G9Bnnbo6Z3JdXr1fQw==",
|
||||
"version": "1.24.3",
|
||||
"resolved": "https://registry.npmjs.org/onnxruntime-node/-/onnxruntime-node-1.24.3.tgz",
|
||||
"integrity": "sha512-JH7+czbc8ALA819vlTgcV+Q214/+VjGeBHDjX81+ZCD0PCVCIFGFNtT0V4sXG/1JXypKPgScQcB3ij/hk3YnTg==",
|
||||
"hasInstallScript": true,
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
@@ -30297,8 +30324,8 @@
|
||||
],
|
||||
"dependencies": {
|
||||
"adm-zip": "^0.5.16",
|
||||
"global-agent": "^4.1.3",
|
||||
"onnxruntime-common": "1.27.0"
|
||||
"global-agent": "^3.0.0",
|
||||
"onnxruntime-common": "1.24.3"
|
||||
}
|
||||
},
|
||||
"node_modules/onnxruntime-web": {
|
||||
@@ -33975,6 +34002,24 @@
|
||||
"url": "https://github.com/sponsors/isaacs"
|
||||
}
|
||||
},
|
||||
"node_modules/roarr": {
|
||||
"version": "2.15.4",
|
||||
"resolved": "https://registry.npmjs.org/roarr/-/roarr-2.15.4.tgz",
|
||||
"integrity": "sha512-CHhPh+UNHD2GTXNYhPWLnU8ONHdI+5DI+4EYIAOaiD63rHeYlZvyh8P+in5999TTSFgUYuKUAjzRI4mdh/p+2A==",
|
||||
"license": "BSD-3-Clause",
|
||||
"optional": true,
|
||||
"dependencies": {
|
||||
"boolean": "^3.0.1",
|
||||
"detect-node": "^2.0.4",
|
||||
"globalthis": "^1.0.1",
|
||||
"json-stringify-safe": "^5.0.1",
|
||||
"semver-compare": "^1.0.0",
|
||||
"sprintf-js": "^1.1.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=8.0"
|
||||
}
|
||||
},
|
||||
"node_modules/robot3": {
|
||||
"version": "0.4.1",
|
||||
"resolved": "https://registry.npmjs.org/robot3/-/robot3-0.4.1.tgz",
|
||||
@@ -34334,6 +34379,13 @@
|
||||
"semver": "bin/semver.js"
|
||||
}
|
||||
},
|
||||
"node_modules/semver-compare": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/semver-compare/-/semver-compare-1.0.0.tgz",
|
||||
"integrity": "sha512-YM3/ITh2MJ5MtzaM429anh+x2jiLVjqILF4m4oyQB18W7Ggea7BfqdH/wGMK7dDiMghv/6WG7znWMwUDzJiXow==",
|
||||
"license": "MIT",
|
||||
"optional": true
|
||||
},
|
||||
"node_modules/send": {
|
||||
"version": "1.2.1",
|
||||
"resolved": "https://registry.npmjs.org/send/-/send-1.2.1.tgz",
|
||||
@@ -34361,13 +34413,13 @@
|
||||
}
|
||||
},
|
||||
"node_modules/serialize-error": {
|
||||
"version": "8.1.0",
|
||||
"resolved": "https://registry.npmjs.org/serialize-error/-/serialize-error-8.1.0.tgz",
|
||||
"integrity": "sha512-3NnuWfM6vBYoy5gZFvHiYsVbafvI9vZv/+jlIigFn4oP4zjNPK3LhcY0xSCgeb1a5L8jO71Mit9LlNoi2UfDDQ==",
|
||||
"version": "7.0.1",
|
||||
"resolved": "https://registry.npmjs.org/serialize-error/-/serialize-error-7.0.1.tgz",
|
||||
"integrity": "sha512-8I8TjW5KMOKsZQTvoxjuSIa7foAwPWGOts+6o7sgjz41/qMD9VQHEDxi6PBvK2l0MXUmqZyNpUK+T2tQaaElvw==",
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"dependencies": {
|
||||
"type-fest": "^0.20.2"
|
||||
"type-fest": "^0.13.1"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=10"
|
||||
@@ -34377,9 +34429,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/serialize-error/node_modules/type-fest": {
|
||||
"version": "0.20.2",
|
||||
"resolved": "https://registry.npmjs.org/type-fest/-/type-fest-0.20.2.tgz",
|
||||
"integrity": "sha512-Ne+eE4r0/iWnpAxD852z3A+N0Bt5RN//NjJwRd2VFHEmrywxf5vsZlh4R6lixl6B+wz/8d+maTSAkN1FIkI3LQ==",
|
||||
"version": "0.13.1",
|
||||
"resolved": "https://registry.npmjs.org/type-fest/-/type-fest-0.13.1.tgz",
|
||||
"integrity": "sha512-34R7HTnG0XIJcBSn5XhDd7nNFPRcXYRZrBB2O2jdKqYODldSzBAqzsWoZYYvduky73toYS/ESqxPvkDf/F0XMg==",
|
||||
"license": "(MIT OR CC0-1.0)",
|
||||
"optional": true,
|
||||
"engines": {
|
||||
@@ -35149,6 +35201,13 @@
|
||||
"node": ">= 10.x"
|
||||
}
|
||||
},
|
||||
"node_modules/sprintf-js": {
|
||||
"version": "1.1.3",
|
||||
"resolved": "https://registry.npmjs.org/sprintf-js/-/sprintf-js-1.1.3.tgz",
|
||||
"integrity": "sha512-Oo+0REFV59/rz3gfJNKQiBlwfHaSESl1pcGyABQsnnIfWOFt6JNj5gCog2U6MLZ//IGYD+nA8nI+mTShREReaA==",
|
||||
"license": "BSD-3-Clause",
|
||||
"optional": true
|
||||
},
|
||||
"node_modules/sql.js": {
|
||||
"version": "1.14.2",
|
||||
"resolved": "https://registry.npmjs.org/sql.js/-/sql.js-1.14.2.tgz",
|
||||
|
||||
@@ -252,7 +252,7 @@
|
||||
"postinstall": "node scripts/build/postinstall.mjs",
|
||||
"uninstall": "node scripts/build/uninstall.mjs",
|
||||
"uninstall:full": "node scripts/build/uninstall.mjs --full",
|
||||
"prepare": "husky",
|
||||
"prepare": "node -e \"try{require.resolve('husky')}catch(e){process.exit(0)};require('child_process').execSync('husky',{stdio:'inherit'})\"",
|
||||
"system-info": "node scripts/dev/system-info.mjs",
|
||||
"build:cli-api": "node --import tsx/esm scripts/cli/generate-api-commands.mjs",
|
||||
"postbuild": "node scripts/build/colocate-standalone.mjs",
|
||||
@@ -347,7 +347,7 @@
|
||||
"better-sqlite3": "^13.0.2",
|
||||
"js-tiktoken": "^1.0.20",
|
||||
"keytar": "^7.9.0",
|
||||
"onnxruntime-node": "1.27.0",
|
||||
"onnxruntime-node": "1.24.3",
|
||||
"sqlite-vec": "^0.1.9",
|
||||
"tls-client-node": "^0.2.0",
|
||||
"wreq-js": "^3.1.0"
|
||||
@@ -436,7 +436,7 @@
|
||||
"unrs-resolver": true
|
||||
},
|
||||
"overrides": {
|
||||
"onnxruntime-node": "1.27.0",
|
||||
"onnxruntime-node": "1.24.3",
|
||||
"fast-xml-parser": "^5.10.1",
|
||||
"sharp": "^0.35.3",
|
||||
"postcss": "^8.5.18",
|
||||
|
||||
@@ -19,12 +19,20 @@
|
||||
*/
|
||||
|
||||
const KEY = process.env.NVIDIA_API_KEY ?? "";
|
||||
const BASE_URL = process.env.NVIDIA_BASE_URL || "https://integrate.api.nvidia.com/v1/chat/completions";
|
||||
const BASE_URL =
|
||||
process.env.NVIDIA_BASE_URL || "https://integrate.api.nvidia.com/v1/chat/completions";
|
||||
const MODEL = process.env.NVIDIA_MODEL || "openai/gpt-oss-120b";
|
||||
|
||||
// Neutralize CR/LF before logging so env-derived values (NVIDIA_MODEL, etc.)
|
||||
// cannot forge extra log lines (S5145 log injection).
|
||||
const line = (s = "") => console.log(String(s).replace(/[\r\n]+/g, " "));
|
||||
// cannot forge extra log lines (S5145 log injection). Also strip any raw
|
||||
// occurrence of the API key so an upstream error/response that echoes it
|
||||
// back (e.g. inside err.stack or a validation result) never reaches the
|
||||
// terminal in clear text (js/clear-text-logging, CWE-312/532).
|
||||
const line = (s = "") => {
|
||||
let out = String(s).replace(/[\r\n]+/g, " ");
|
||||
if (KEY) out = out.split(KEY).join("[REDACTED]");
|
||||
console.log(out);
|
||||
};
|
||||
const hr = () => line("─".repeat(72));
|
||||
|
||||
function show(label: string, value: unknown) {
|
||||
@@ -52,8 +60,13 @@ async function partA() {
|
||||
});
|
||||
line(" ✅ validateProviderApiKey retornou (sem crash):");
|
||||
show("resultado", result);
|
||||
if (typeof (result as any)?.error === "string" && (result as any).error.includes("startsWith")) {
|
||||
line(" ⚠️ A mensagem de erro contém 'startsWith' → crash CAPTURADO dentro do try/catch da validação.");
|
||||
if (
|
||||
typeof (result as any)?.error === "string" &&
|
||||
(result as any).error.includes("startsWith")
|
||||
) {
|
||||
line(
|
||||
" ⚠️ A mensagem de erro contém 'startsWith' → crash CAPTURADO dentro do try/catch da validação."
|
||||
);
|
||||
}
|
||||
} catch (err: any) {
|
||||
line(" ❌ validateProviderApiKey LANÇOU (crash não tratado):");
|
||||
|
||||
@@ -23,7 +23,7 @@ import {
|
||||
statSync,
|
||||
chmodSync,
|
||||
} from "node:fs";
|
||||
import { join, dirname } from "node:path";
|
||||
import { join, dirname, relative } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
import { assembleStandalone } from "./assembleStandalone.mjs";
|
||||
@@ -35,6 +35,12 @@ import {
|
||||
APP_STAGING_REMOVAL_PATHS,
|
||||
findUnexpectedArtifactPaths,
|
||||
} from "./pack-artifact-policy.ts";
|
||||
import {
|
||||
collectWorkspaceVersions,
|
||||
findPackageJsonFiles,
|
||||
hasWorkspaceProtocol,
|
||||
resolvePackageJsonWorkspaceProtocols,
|
||||
} from "./resolveWorkspaceProtocols.ts";
|
||||
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const __dirname = dirname(__filename);
|
||||
@@ -707,6 +713,33 @@ if (remainingUnexpectedFiles.length > 0) {
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// -- Step 11: Resolve workspace: protocol dependencies -----------------
|
||||
// npm/pnpm workspace protocol specifiers (workspace:*, workspace:^, ...)
|
||||
// are meaningless to the npm registry and make `npm install -g omniroute`
|
||||
// fail with EUNSUPPORTEDPROTOCOL. Rewrite any that leaked into published
|
||||
// package.json files to the concrete workspace package version.
|
||||
// Only touch files inside the staged dist/ tree; workspace member source
|
||||
// package.json files must never be mutated by the publish step.
|
||||
const workspaceVersions = collectWorkspaceVersions(ROOT);
|
||||
const publishablePackageJsonDirs = [DIST_DIR];
|
||||
const publishablePackageJsonPaths = publishablePackageJsonDirs
|
||||
.flatMap((dir) => (existsSync(dir) ? findPackageJsonFiles(dir) : []))
|
||||
.filter((filePath) => existsSync(filePath));
|
||||
|
||||
for (const pkgJsonPath of publishablePackageJsonPaths) {
|
||||
let pkg: Record<string, unknown>;
|
||||
try {
|
||||
pkg = JSON.parse(readFileSync(pkgJsonPath, "utf8")) as Record<string, unknown>;
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
if (!hasWorkspaceProtocol(pkg)) continue;
|
||||
|
||||
const resolved = resolvePackageJsonWorkspaceProtocols(pkg, workspaceVersions);
|
||||
writeFileSync(pkgJsonPath, JSON.stringify(resolved, null, 2) + "\n");
|
||||
console.log(` [resolved] Resolved workspace: protocols in ${relative(ROOT, pkgJsonPath)}`);
|
||||
}
|
||||
|
||||
// ── Done ───────────────────────────────────────────────────
|
||||
const distPkg = join(DIST_DIR, "package.json");
|
||||
if (existsSync(distPkg)) {
|
||||
|
||||
228
scripts/build/resolveWorkspaceProtocols.ts
Normal file
228
scripts/build/resolveWorkspaceProtocols.ts
Normal file
@@ -0,0 +1,228 @@
|
||||
/**
|
||||
* Resolve pnpm/npm workspace protocol dependencies to concrete semver versions.
|
||||
*
|
||||
* The npm registry clients cannot parse `workspace:` specifiers. During prepublish
|
||||
* we rewrite any `workspace:*`, `workspace:^`, `workspace:~` (or explicit
|
||||
* `workspace:<range>`) dependency declarations to the matching workspace package's
|
||||
* actual version before npm pack/publish sees them.
|
||||
*/
|
||||
|
||||
import { readFileSync, readdirSync, statSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import * as yaml from "js-yaml";
|
||||
|
||||
const WORKSPACE_PROTOCOL_RE = /^workspace:/;
|
||||
|
||||
const DEPENDENCY_FIELDS = [
|
||||
"dependencies",
|
||||
"devDependencies",
|
||||
"peerDependencies",
|
||||
"optionalDependencies",
|
||||
] as const;
|
||||
|
||||
/**
|
||||
* Parse a simple workspace glob entry into concrete directories relative to a root.
|
||||
* Supports entries like "packages/*" and literal directory names like "open-sse".
|
||||
*/
|
||||
function expandWorkspaceEntry(root: string, entry: string): string[] {
|
||||
const trimmed = entry.trim();
|
||||
if (!trimmed) return [];
|
||||
if (!trimmed.endsWith("/*")) {
|
||||
const dir = join(root, trimmed);
|
||||
try {
|
||||
return statSync(dir).isDirectory() ? [dir] : [];
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
const parent = join(root, trimmed.slice(0, -2));
|
||||
let entries: string[] = [];
|
||||
try {
|
||||
entries = readdirSync(parent);
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
return entries
|
||||
.map((name) => join(parent, name))
|
||||
.filter((dir) => {
|
||||
try {
|
||||
return statSync(dir).isDirectory();
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Read the root package.json and, if present, pnpm-workspace.yaml to discover
|
||||
* workspace member directories. Returns a map of package name -> version.
|
||||
*/
|
||||
export function collectWorkspaceVersions(projectRoot: string): Map<string, string> {
|
||||
const versions = new Map<string, string>();
|
||||
|
||||
const rootPkgPath = join(projectRoot, "package.json");
|
||||
let workspaceEntries: string[] = [];
|
||||
try {
|
||||
const rootPkg = JSON.parse(readFileSync(rootPkgPath, "utf8")) as {
|
||||
workspaces?: string[];
|
||||
};
|
||||
if (Array.isArray(rootPkg.workspaces)) {
|
||||
workspaceEntries.push(...rootPkg.workspaces);
|
||||
}
|
||||
} catch {
|
||||
// ignore unreadable root package.json
|
||||
}
|
||||
|
||||
const pnpmWorkspacePath = join(projectRoot, "pnpm-workspace.yaml");
|
||||
try {
|
||||
const yamlContent = readFileSync(pnpmWorkspacePath, "utf8");
|
||||
const doc = yaml.load(yamlContent) as { packages?: unknown } | null | undefined;
|
||||
if (doc && Array.isArray(doc.packages)) {
|
||||
for (const entry of doc.packages) {
|
||||
if (typeof entry === "string" && entry) {
|
||||
workspaceEntries.push(entry);
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// ignore missing or malformed pnpm-workspace.yaml
|
||||
}
|
||||
|
||||
const seenDirs = new Set<string>();
|
||||
for (const entry of workspaceEntries) {
|
||||
for (const dir of expandWorkspaceEntry(projectRoot, entry)) {
|
||||
if (seenDirs.has(dir)) continue;
|
||||
seenDirs.add(dir);
|
||||
try {
|
||||
const pkg = JSON.parse(readFileSync(join(dir, "package.json"), "utf8")) as {
|
||||
name?: string;
|
||||
version?: string;
|
||||
};
|
||||
if (pkg.name && pkg.version) {
|
||||
versions.set(pkg.name, pkg.version);
|
||||
}
|
||||
} catch {
|
||||
// skip unreadable workspace member package.json
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return versions;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve workspace protocol dependencies inside a package.json object.
|
||||
*
|
||||
* Replaces `workspace:*`, `workspace:^`, `workspace:~`, `workspace:<range>`,
|
||||
* and `workspace:<packageName>` with the concrete version of the referenced
|
||||
* workspace package. Throws if a workspace specifier cannot be resolved.
|
||||
*/
|
||||
export function resolvePackageJsonWorkspaceProtocols(
|
||||
pkg: Record<string, unknown>,
|
||||
workspaceVersions: Map<string, string>
|
||||
): Record<string, unknown> {
|
||||
const resolved: Record<string, unknown> = { ...pkg };
|
||||
|
||||
for (const field of DEPENDENCY_FIELDS) {
|
||||
const deps = pkg[field];
|
||||
if (!deps || typeof deps !== "object" || Array.isArray(deps)) continue;
|
||||
|
||||
const resolvedDeps: Record<string, string> = {};
|
||||
let changed = false;
|
||||
for (const [depName, versionSpec] of Object.entries(deps as Record<string, unknown>)) {
|
||||
if (typeof versionSpec !== "string") {
|
||||
resolvedDeps[depName] = String(versionSpec ?? "");
|
||||
continue;
|
||||
}
|
||||
if (!WORKSPACE_PROTOCOL_RE.test(versionSpec)) {
|
||||
resolvedDeps[depName] = versionSpec;
|
||||
continue;
|
||||
}
|
||||
|
||||
const body = versionSpec.slice("workspace:".length);
|
||||
let concrete: string | undefined;
|
||||
|
||||
if (body === "*") {
|
||||
concrete = workspaceVersions.get(depName);
|
||||
} else if (body === "^") {
|
||||
const version = workspaceVersions.get(depName);
|
||||
concrete = version ? `^${version}` : undefined;
|
||||
} else if (body === "~") {
|
||||
const version = workspaceVersions.get(depName);
|
||||
concrete = version ? `~${version}` : undefined;
|
||||
} else if (body.startsWith("^") || body.startsWith("~") || /^[\d<>=]/.test(body)) {
|
||||
// Explicit range inside workspace: protocol - strip the protocol prefix.
|
||||
concrete = body;
|
||||
} else {
|
||||
// workspace:<packageName> - resolve to that package's version.
|
||||
concrete = workspaceVersions.get(body);
|
||||
}
|
||||
|
||||
if (concrete) {
|
||||
resolvedDeps[depName] = concrete;
|
||||
changed = true;
|
||||
} else {
|
||||
throw new Error(
|
||||
`Cannot resolve workspace protocol "${versionSpec}" for dependency "${depName}". ` +
|
||||
"Make sure the referenced package is a declared workspace member with a version."
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
if (changed) {
|
||||
resolved[field] = resolvedDeps;
|
||||
}
|
||||
}
|
||||
|
||||
return resolved;
|
||||
}
|
||||
|
||||
/**
|
||||
* Return true if any dependency field in the package contains a workspace: specifier.
|
||||
*/
|
||||
export function hasWorkspaceProtocol(pkg: Record<string, unknown>): boolean {
|
||||
for (const field of DEPENDENCY_FIELDS) {
|
||||
const deps = pkg[field];
|
||||
if (!deps || typeof deps !== "object" || Array.isArray(deps)) continue;
|
||||
for (const versionSpec of Object.values(deps as Record<string, unknown>)) {
|
||||
if (typeof versionSpec === "string" && WORKSPACE_PROTOCOL_RE.test(versionSpec)) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Recursively walk a directory and return every package.json path found.
|
||||
* Stops descending after maxDepth to avoid runaway recursion on deep trees.
|
||||
*/
|
||||
export function findPackageJsonFiles(dir: string, maxDepth = 10): string[] {
|
||||
const results: string[] = [];
|
||||
if (maxDepth < 0) return results;
|
||||
|
||||
let entries: string[] = [];
|
||||
try {
|
||||
entries = readdirSync(dir);
|
||||
} catch {
|
||||
return results;
|
||||
}
|
||||
|
||||
for (const entry of entries) {
|
||||
if (entry === "node_modules") continue;
|
||||
const fullPath = join(dir, entry);
|
||||
let stat;
|
||||
try {
|
||||
stat = statSync(fullPath);
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
if (stat.isDirectory()) {
|
||||
results.push(...findPackageJsonFiles(fullPath, maxDepth - 1));
|
||||
} else if (entry === "package.json") {
|
||||
results.push(fullPath);
|
||||
}
|
||||
}
|
||||
|
||||
return results;
|
||||
}
|
||||
@@ -66,8 +66,8 @@ export function structuralRejectionResponse(status: 413 | 503, maxMessages: numb
|
||||
{
|
||||
type: historyLimit ? "payload_too_large" : "server_error",
|
||||
code: historyLimit ? "chat_history_too_large" : "chat_admission_busy",
|
||||
reason: historyLimit ? "message_limit" : "structure_limit",
|
||||
}
|
||||
);
|
||||
body.error.reason = historyLimit ? "message_limit" : "structure_limit";
|
||||
return new Response(JSON.stringify(body), { status, headers });
|
||||
}
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { describe, it, before } from "node:test";
|
||||
import { describe, it, beforeAll } from "vitest";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import { getDbInstance } from "../../../src/lib/db/core.ts";
|
||||
@@ -51,7 +51,7 @@ function seed(provider: string, modelId: string, caps: {
|
||||
).run(provider, modelId, caps.tool_call ?? null, caps.reasoning ?? null, caps.limit_context ?? null);
|
||||
}
|
||||
|
||||
before(() => {
|
||||
beforeAll(() => {
|
||||
ensureTable();
|
||||
// Force both module caches to re-read after our seeds.
|
||||
invalidateCapabilitiesCache();
|
||||
|
||||
269
tests/unit/build/pack-no-workspace-protocol.test.ts
Normal file
269
tests/unit/build/pack-no-workspace-protocol.test.ts
Normal file
@@ -0,0 +1,269 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
import {
|
||||
collectWorkspaceVersions,
|
||||
resolvePackageJsonWorkspaceProtocols,
|
||||
hasWorkspaceProtocol,
|
||||
findPackageJsonFiles,
|
||||
} from "../../../scripts/build/resolveWorkspaceProtocols.ts";
|
||||
|
||||
function tmpDir(prefix: string): string {
|
||||
return fs.mkdtempSync(path.join(os.tmpdir(), prefix));
|
||||
}
|
||||
|
||||
function writeJson(filePath: string, data: unknown): void {
|
||||
fs.mkdirSync(path.dirname(filePath), { recursive: true });
|
||||
fs.writeFileSync(filePath, JSON.stringify(data, null, 2));
|
||||
}
|
||||
|
||||
function readJson(filePath: string): Record<string, unknown> {
|
||||
return JSON.parse(fs.readFileSync(filePath, "utf8")) as Record<string, unknown>;
|
||||
}
|
||||
|
||||
test("resolvePackageJsonWorkspaceProtocols replaces workspace:*, workspace:^, workspace:~", () => {
|
||||
const versions = new Map([
|
||||
["@omniroute/open-sse", "3.8.51"],
|
||||
["@omniroute/shared", "1.2.3"],
|
||||
]);
|
||||
|
||||
const resolved = resolvePackageJsonWorkspaceProtocols(
|
||||
{
|
||||
name: "omniroute",
|
||||
version: "3.8.51",
|
||||
dependencies: {
|
||||
"@omniroute/open-sse": "workspace:^",
|
||||
"@omniroute/shared": "workspace:*",
|
||||
lodash: "^4.17.0",
|
||||
},
|
||||
devDependencies: {
|
||||
"@omniroute/open-sse": "workspace:~",
|
||||
},
|
||||
peerDependencies: {
|
||||
"@omniroute/shared": "workspace:1.2.3",
|
||||
},
|
||||
optionalDependencies: {
|
||||
"@omniroute/open-sse": "workspace:>=3.0.0",
|
||||
},
|
||||
},
|
||||
versions
|
||||
);
|
||||
|
||||
assert.equal((resolved.dependencies as Record<string, string>)["@omniroute/open-sse"], "^3.8.51");
|
||||
assert.equal((resolved.dependencies as Record<string, string>)["@omniroute/shared"], "1.2.3");
|
||||
assert.equal((resolved.dependencies as Record<string, string>).lodash, "^4.17.0");
|
||||
assert.equal(
|
||||
(resolved.devDependencies as Record<string, string>)["@omniroute/open-sse"],
|
||||
"~3.8.51"
|
||||
);
|
||||
assert.equal((resolved.peerDependencies as Record<string, string>)["@omniroute/shared"], "1.2.3");
|
||||
assert.equal(
|
||||
(resolved.optionalDependencies as Record<string, string>)["@omniroute/open-sse"],
|
||||
">=3.0.0"
|
||||
);
|
||||
});
|
||||
|
||||
test("resolvePackageJsonWorkspaceProtocols leaves non-workspace specs untouched", () => {
|
||||
const resolved = resolvePackageJsonWorkspaceProtocols(
|
||||
{
|
||||
name: "x",
|
||||
dependencies: {
|
||||
a: "^1.0.0",
|
||||
b: "file:../b",
|
||||
c: "npm:alias@1.0.0",
|
||||
},
|
||||
},
|
||||
new Map()
|
||||
);
|
||||
|
||||
assert.equal((resolved.dependencies as Record<string, string>).a, "^1.0.0");
|
||||
assert.equal((resolved.dependencies as Record<string, string>).b, "file:../b");
|
||||
assert.equal((resolved.dependencies as Record<string, string>).c, "npm:alias@1.0.0");
|
||||
assert.equal(hasWorkspaceProtocol(resolved), false);
|
||||
});
|
||||
|
||||
test("resolvePackageJsonWorkspaceProtocols throws for unresolvable workspace protocol", () => {
|
||||
assert.throws(
|
||||
() =>
|
||||
resolvePackageJsonWorkspaceProtocols(
|
||||
{
|
||||
name: "x",
|
||||
dependencies: {
|
||||
"@missing/pkg": "workspace:^",
|
||||
},
|
||||
},
|
||||
new Map()
|
||||
),
|
||||
/Cannot resolve workspace protocol/
|
||||
);
|
||||
});
|
||||
|
||||
test("collectWorkspaceVersions reads npm workspaces and pnpm-workspace.yaml", () => {
|
||||
const root = tmpDir("workspace-versions-");
|
||||
|
||||
writeJson(path.join(root, "package.json"), {
|
||||
name: "root",
|
||||
version: "0.0.0",
|
||||
workspaces: ["packages/*", "open-sse"],
|
||||
});
|
||||
|
||||
fs.mkdirSync(path.join(root, "packages", "a"), { recursive: true });
|
||||
writeJson(path.join(root, "packages", "a", "package.json"), {
|
||||
name: "@scope/a",
|
||||
version: "1.0.0",
|
||||
});
|
||||
|
||||
fs.mkdirSync(path.join(root, "open-sse"), { recursive: true });
|
||||
writeJson(path.join(root, "open-sse", "package.json"), {
|
||||
name: "@scope/open-sse",
|
||||
version: "2.0.0",
|
||||
});
|
||||
|
||||
// pnpm-workspace.yaml adds an extra directory not in npm workspaces.
|
||||
fs.mkdirSync(path.join(root, "packages", "b"), { recursive: true });
|
||||
writeJson(path.join(root, "packages", "b", "package.json"), {
|
||||
name: "@scope/b",
|
||||
version: "3.0.0",
|
||||
});
|
||||
fs.writeFileSync(path.join(root, "pnpm-workspace.yaml"), "packages:\n - 'packages/*'\n");
|
||||
|
||||
const versions = collectWorkspaceVersions(root);
|
||||
assert.equal(versions.get("@scope/a"), "1.0.0");
|
||||
assert.equal(versions.get("@scope/open-sse"), "2.0.0");
|
||||
assert.equal(versions.get("@scope/b"), "3.0.0");
|
||||
});
|
||||
|
||||
test("findPackageJsonFiles skips node_modules and respects maxDepth", () => {
|
||||
const root = tmpDir("pkg-json-files-");
|
||||
fs.mkdirSync(path.join(root, "a"), { recursive: true });
|
||||
writeJson(path.join(root, "a", "package.json"), {});
|
||||
fs.mkdirSync(path.join(root, "node_modules", "x"), { recursive: true });
|
||||
writeJson(path.join(root, "node_modules", "x", "package.json"), {});
|
||||
|
||||
const files = findPackageJsonFiles(root);
|
||||
assert.equal(files.length, 1);
|
||||
assert.ok(files[0].endsWith(path.join("a", "package.json")));
|
||||
|
||||
// Build a deep tree and confirm maxDepth bounds the walk.
|
||||
const deep = tmpDir("pkg-json-deep-");
|
||||
let current = deep;
|
||||
for (let i = 0; i < 12; i += 1) {
|
||||
current = path.join(current, `level${i}`);
|
||||
fs.mkdirSync(current, { recursive: true });
|
||||
}
|
||||
writeJson(path.join(current, "package.json"), {});
|
||||
assert.equal(findPackageJsonFiles(deep, 10).length, 0);
|
||||
assert.equal(findPackageJsonFiles(deep, 12).length, 1);
|
||||
});
|
||||
|
||||
test("collectWorkspaceVersions parses pnpm-workspace.yaml with js-yaml", () => {
|
||||
const root = tmpDir("pnpm-yaml-");
|
||||
|
||||
writeJson(path.join(root, "package.json"), { name: "root", version: "0.0.0" });
|
||||
|
||||
// Flow-style array, nested quotes, comments inside the packages list, and an
|
||||
// unrelated top-level key before packages are all valid YAML that the old line
|
||||
// scanner could not handle.
|
||||
fs.writeFileSync(
|
||||
path.join(root, "pnpm-workspace.yaml"),
|
||||
"preferWorkspacePackages: true\n" +
|
||||
"packages:\n" +
|
||||
' - "packages/*"\n' +
|
||||
" - 'apps/*'\n" +
|
||||
" # comment inside the list\n" +
|
||||
" - open-sse\n"
|
||||
);
|
||||
|
||||
fs.mkdirSync(path.join(root, "packages", "a"), { recursive: true });
|
||||
writeJson(path.join(root, "packages", "a", "package.json"), {
|
||||
name: "@scope/a",
|
||||
version: "1.0.0",
|
||||
});
|
||||
|
||||
fs.mkdirSync(path.join(root, "apps", "web"), { recursive: true });
|
||||
writeJson(path.join(root, "apps", "web", "package.json"), {
|
||||
name: "@scope/web",
|
||||
version: "2.0.0",
|
||||
});
|
||||
|
||||
fs.mkdirSync(path.join(root, "open-sse"), { recursive: true });
|
||||
writeJson(path.join(root, "open-sse", "package.json"), {
|
||||
name: "@scope/open-sse",
|
||||
version: "3.0.0",
|
||||
});
|
||||
|
||||
const versions = collectWorkspaceVersions(root);
|
||||
assert.equal(versions.get("@scope/a"), "1.0.0");
|
||||
assert.equal(versions.get("@scope/web"), "2.0.0");
|
||||
assert.equal(versions.get("@scope/open-sse"), "3.0.0");
|
||||
});
|
||||
|
||||
test("prepublish Step 11 fixture resolves workspace: protocols in dist package.json files", () => {
|
||||
const root = tmpDir("prepublish-step11-");
|
||||
const distDir = path.join(root, "dist");
|
||||
|
||||
// Workspace member source files contain a workspace: specifier (simulating the
|
||||
// monorepo source). They must NOT be mutated by the publish step.
|
||||
fs.mkdirSync(path.join(root, "packages", "shared"), { recursive: true });
|
||||
const sourcePkgPath = path.join(root, "packages", "shared", "package.json");
|
||||
writeJson(sourcePkgPath, {
|
||||
name: "@scope/shared",
|
||||
version: "1.2.3",
|
||||
dependencies: {
|
||||
"@scope/other": "workspace:*",
|
||||
},
|
||||
});
|
||||
|
||||
fs.mkdirSync(path.join(root, "packages", "other"), { recursive: true });
|
||||
writeJson(path.join(root, "packages", "other", "package.json"), {
|
||||
name: "@scope/other",
|
||||
version: "4.5.6",
|
||||
});
|
||||
|
||||
writeJson(path.join(root, "package.json"), {
|
||||
name: "root",
|
||||
version: "0.0.0",
|
||||
workspaces: ["packages/*"],
|
||||
});
|
||||
|
||||
// The staged dist/ package.json contains workspace: specifiers that leaked
|
||||
// into the publish artifact and must be rewritten to concrete versions.
|
||||
fs.mkdirSync(distDir, { recursive: true });
|
||||
const distPkgPath = path.join(distDir, "package.json");
|
||||
writeJson(distPkgPath, {
|
||||
name: "omniroute",
|
||||
version: "3.8.51",
|
||||
dependencies: {
|
||||
"@scope/shared": "workspace:^",
|
||||
"@scope/other": "workspace:*",
|
||||
lodash: "^4.17.0",
|
||||
},
|
||||
});
|
||||
|
||||
// This is the same logic prepublish.ts Step 11 runs, scoped to the fixture.
|
||||
const workspaceVersions = collectWorkspaceVersions(root);
|
||||
const publishablePackageJsonPaths = findPackageJsonFiles(distDir).filter((filePath) =>
|
||||
fs.existsSync(filePath)
|
||||
);
|
||||
|
||||
for (const pkgJsonPath of publishablePackageJsonPaths) {
|
||||
const pkg = readJson(pkgJsonPath);
|
||||
if (!hasWorkspaceProtocol(pkg)) continue;
|
||||
const resolved = resolvePackageJsonWorkspaceProtocols(pkg, workspaceVersions);
|
||||
fs.writeFileSync(pkgJsonPath, JSON.stringify(resolved, null, 2) + "\n");
|
||||
}
|
||||
|
||||
// dist/package.json must have concrete versions.
|
||||
const distPkg = readJson(distPkgPath);
|
||||
assert.equal((distPkg.dependencies as Record<string, string>)["@scope/shared"], "^1.2.3");
|
||||
assert.equal((distPkg.dependencies as Record<string, string>)["@scope/other"], "4.5.6");
|
||||
assert.equal((distPkg.dependencies as Record<string, string>).lodash, "^4.17.0");
|
||||
assert.equal(hasWorkspaceProtocol(distPkg), false);
|
||||
|
||||
// Source package.json must remain untouched.
|
||||
const sourcePkg = readJson(sourcePkgPath);
|
||||
assert.equal((sourcePkg.dependencies as Record<string, string>)["@scope/other"], "workspace:*");
|
||||
});
|
||||
35
tests/unit/chat-admission-rejection-reason.test.ts
Normal file
35
tests/unit/chat-admission-rejection-reason.test.ts
Normal file
@@ -0,0 +1,35 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import {
|
||||
structuralRejectionResponse,
|
||||
} from "../../src/shared/middleware/chatAdmissionResponses.ts";
|
||||
|
||||
// Pins the machine-readable error.reason contract of the chat admission
|
||||
// structural rejections (#TS2339 regression guard): buildErrorBody now owns
|
||||
// the reason field via ErrorBodyClassification, so the response bodies keep
|
||||
// carrying it without post-construction mutation of an untyped field.
|
||||
test("structuralRejectionResponse 413 carries reason=message_limit classification", () => {
|
||||
const res = structuralRejectionResponse(413, 40);
|
||||
assert.equal(res.status, 413);
|
||||
assert.ok(!res.headers.has("Retry-After"), "413 is not retryable-by-header");
|
||||
return res.text().then((raw) => {
|
||||
const body = JSON.parse(raw);
|
||||
assert.equal(body.error.reason, "message_limit");
|
||||
assert.equal(body.error.type, "payload_too_large");
|
||||
assert.equal(body.error.code, "chat_history_too_large");
|
||||
assert.ok(!body.error.message.includes("at /"), "must not leak stack traces");
|
||||
});
|
||||
});
|
||||
|
||||
test("structuralRejectionResponse 503 carries reason=structure_limit and Retry-After", () => {
|
||||
const res = structuralRejectionResponse(503, 40);
|
||||
assert.equal(res.status, 503);
|
||||
assert.equal(res.headers.get("Retry-After"), "1");
|
||||
return res.text().then((raw) => {
|
||||
const body = JSON.parse(raw);
|
||||
assert.equal(body.error.reason, "structure_limit");
|
||||
assert.equal(body.error.type, "server_error");
|
||||
assert.equal(body.error.code, "chat_admission_busy");
|
||||
});
|
||||
});
|
||||
@@ -4,28 +4,37 @@ import { readFileSync } from "node:fs";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
// The Docker publish workflow builds on GitHub-hosted runners (ubuntu-24.04 and
|
||||
// ubuntu-24.04-arm): 4 vCPU, 16 GB RAM. Every Next page-data worker is its own
|
||||
// process and inherits NODE_OPTIONS, so the V8 ceiling is per PROCESS: the
|
||||
// build's worst case is roughly `workers × OMNIROUTE_BUILD_MEMORY_MB`.
|
||||
// ubuntu-24.04-arm): 4 vCPU, 16 GB RAM. Every Next page-data worker AND the
|
||||
// parent `next build` process are separate OS processes, so the budget has to
|
||||
// cover all of them, not just the workers.
|
||||
//
|
||||
// With 7 workers × 6144 MB the runner ran out and buildkit failed the step with
|
||||
// `ResourceExhausted: ... cannot allocate memory`, right after "Collecting page
|
||||
// data using 7 workers" — every Docker publish since 2026-08-22 23:14 UTC.
|
||||
// Lowering to 2 workers (#10060 / PR #11419) was not enough: it modeled the
|
||||
// per-process peak as an INFERENCE (`WORKER_PEAK_MB = 2560`, derived only from
|
||||
// "7 workers didn't fit") and assumed the parent process tracked the V8 heap
|
||||
// ceiling (`OMNIROUTE_BUILD_MEMORY_MB`) rather than its own RSS. The owner's
|
||||
// live VPS reproduction (issue #7518, dmesg OOM-killer report, 2026-08-24)
|
||||
// measured the real number directly: `next-build (v16) ... anon-rss:4522744kB`
|
||||
// (~4.5 GB) per process, independent of the NODE_OPTIONS heap flag — Turbopack
|
||||
// itself is native/Rust and compiles outside the V8 heap. With 2 workers that
|
||||
// keeps the publish pipeline failing at "Collecting page data using 2 workers"
|
||||
// (run 32907937950, 2026-08-25).
|
||||
//
|
||||
// This pins the budget so raising either knob has to be a deliberate change
|
||||
// that re-does the arithmetic, not a one-line bump that silently reds the
|
||||
// publish pipeline again.
|
||||
// This pins the budget on the MEASURED figure, applied uniformly to every
|
||||
// process (parent + workers), so raising the worker count has to be a
|
||||
// deliberate change that re-does the arithmetic, not a one-line bump that
|
||||
// silently reds the publish pipeline again.
|
||||
|
||||
const RUNNER_MEMORY_MB = 16 * 1024;
|
||||
// Leave room for buildkit, the snapshotter and page cache.
|
||||
const HEADROOM_FRACTION = 0.75;
|
||||
// Planning figure for one page-data worker's peak RSS. It is an INFERENCE, not
|
||||
// a measurement: 7 workers did not fit in 16 GB alongside the parent, which
|
||||
// puts the per-worker peak somewhere north of ~1.8 GB. 2.5 GB is that bound
|
||||
// rounded up, so the budget below stays conservative. If a future build OOMs
|
||||
// again with a worker count this test accepts, raise this number — do not
|
||||
// weaken the budget.
|
||||
const WORKER_PEAK_MB = 2560;
|
||||
// Measured (not inferred) peak RSS for a single Next/Turbopack build process —
|
||||
// parent or page-data worker alike — from the dmesg OOM-killer report above.
|
||||
// If a future build OOMs again, re-measure via dmesg before raising this
|
||||
// number — do not weaken the budget with another guess.
|
||||
const MEASURED_PROCESS_RSS_MB = 4500;
|
||||
|
||||
const dockerfile = readFileSync(
|
||||
fileURLToPath(new URL("../../Dockerfile", import.meta.url)),
|
||||
@@ -51,25 +60,27 @@ test("the Docker build's worker pool is derived from OMNIROUTE_BUILD_WORKERS", (
|
||||
);
|
||||
});
|
||||
|
||||
test("worker count × per-process heap fits a 16 GB GitHub runner", () => {
|
||||
test("worker count × measured per-process RSS fits a 16 GB GitHub runner", () => {
|
||||
const workerPool = readArgDefault("OMNIROUTE_BUILD_WORKERS");
|
||||
const heapMb = readArgDefault("OMNIROUTE_BUILD_MEMORY_MB");
|
||||
|
||||
// Next derives `workers = CIRCLE_NODE_TOTAL - 1`.
|
||||
const workers = workerPool - 1;
|
||||
assert.ok(workers >= 1, `CIRCLE_NODE_TOTAL=${workerPool} leaves no build workers`);
|
||||
|
||||
// The parent `next build` process is the one that genuinely needs the raised
|
||||
// ceiling (the webpack/turbopack production pass, #4076); the workers are
|
||||
// budgeted at their inferred peak instead.
|
||||
const worstCaseMb = heapMb + workers * WORKER_PEAK_MB;
|
||||
// Every process — the parent `next build` process AND each page-data
|
||||
// worker — is budgeted at the measured per-process RSS floor (see the file
|
||||
// banner comment). The V8 heap ceiling (OMNIROUTE_BUILD_MEMORY_MB) bounds
|
||||
// JS allocations but not Turbopack's native/Rust memory, so it cannot stand
|
||||
// in for the parent process's real RSS.
|
||||
const processes = workers + 1;
|
||||
const worstCaseMb = processes * MEASURED_PROCESS_RSS_MB;
|
||||
const budgetMb = RUNNER_MEMORY_MB * HEADROOM_FRACTION;
|
||||
assert.ok(
|
||||
worstCaseMb <= budgetMb,
|
||||
`parent ${heapMb} MB + ${workers} workers × ${WORKER_PEAK_MB} MB = ${worstCaseMb} MB ` +
|
||||
`exceeds the ${budgetMb} MB budget on a ${RUNNER_MEMORY_MB} MB runner — the Docker ` +
|
||||
`publish step dies with "ResourceExhausted: cannot allocate memory" during page-data ` +
|
||||
`collection`
|
||||
`${processes} processes (1 parent + ${workers} workers) × ${MEASURED_PROCESS_RSS_MB} MB ` +
|
||||
`measured RSS = ${worstCaseMb} MB exceeds the ${budgetMb} MB budget on a ` +
|
||||
`${RUNNER_MEMORY_MB} MB runner — the Docker publish step dies with "ResourceExhausted: ` +
|
||||
`cannot allocate memory" during page-data collection`
|
||||
);
|
||||
});
|
||||
|
||||
|
||||
54
tests/unit/prepare-script-husky-guard.test.ts
Normal file
54
tests/unit/prepare-script-husky-guard.test.ts
Normal file
@@ -0,0 +1,54 @@
|
||||
import { describe, it } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { execSync } from "node:child_process";
|
||||
import { join } from "node:path";
|
||||
|
||||
describe("package.json prepare script (#11571)", () => {
|
||||
const pkg = JSON.parse(
|
||||
readFileSync(join(import.meta.dirname, "../../package.json"), "utf8")
|
||||
);
|
||||
|
||||
it("has a prepare script that guards against missing husky", () => {
|
||||
const prepare = pkg.scripts?.prepare;
|
||||
assert.ok(prepare, "prepare script must exist");
|
||||
assert.ok(
|
||||
prepare.includes("require.resolve") || prepare.includes("existsSync"),
|
||||
"prepare must check if husky is available before running it"
|
||||
);
|
||||
});
|
||||
|
||||
it("does not hard-fail when husky is absent", () => {
|
||||
const prepare = pkg.scripts?.prepare;
|
||||
assert.ok(
|
||||
!prepare.match(/^\s*husky\s*$/),
|
||||
"prepare must not be a bare 'husky' call without a guard"
|
||||
);
|
||||
});
|
||||
|
||||
it("exits cleanly without invoking husky when it is unresolvable", () => {
|
||||
const prepare = pkg.scripts?.prepare;
|
||||
assert.ok(prepare);
|
||||
assert.ok(
|
||||
!prepare.includes("&& husky") && !prepare.includes("|| husky"),
|
||||
"husky must not appear as a separate shell command after the guard — " +
|
||||
"process.exit(0) in the guard would still allow && to proceed"
|
||||
);
|
||||
});
|
||||
|
||||
it("still calls husky when available", () => {
|
||||
const prepare = pkg.scripts?.prepare;
|
||||
assert.ok(
|
||||
prepare.includes("husky"),
|
||||
"prepare must still invoke husky when it is installed"
|
||||
);
|
||||
});
|
||||
|
||||
it("exits 0 in an environment where husky is not resolvable", () => {
|
||||
const result = execSync(
|
||||
"node -e \"try{require.resolve('husky_nonexistent_pkg')}catch(e){process.exit(0)};process.exit(1)\"",
|
||||
{ encoding: "utf8", stdio: "pipe" }
|
||||
);
|
||||
assert.equal(result, "", "should produce no output and exit 0");
|
||||
});
|
||||
});
|
||||
20
tests/unit/zai-web-attachment-mime-contract.test.ts
Normal file
20
tests/unit/zai-web-attachment-mime-contract.test.ts
Normal file
@@ -0,0 +1,20 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import { resolveCursorImages } from "../../open-sse/utils/cursorImages.ts";
|
||||
|
||||
// zai-web maps resolveCursorImages() output into browser-upload attachments
|
||||
// whose mimeType is REQUIRED. EncodedImage.mimeType is optional on the wire
|
||||
// type, so zai-web carries an `?? "image/jpeg"` fallback — this test pins the
|
||||
// producer contract that makes the fallback dead code in practice: every
|
||||
// image that reaches a browser upload must arrive with a concrete image/*
|
||||
// mime string (decodeDataUrl / fetchImageBytes validate it before pushing).
|
||||
const PIXEL_PNG =
|
||||
"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg==";
|
||||
|
||||
test("resolveCursorImages (prepareForWire:false) always yields a concrete image/* mimeType", async () => {
|
||||
const images = await resolveCursorImages([PIXEL_PNG], { prepareForWire: false });
|
||||
assert.equal(images.length, 1);
|
||||
assert.equal(typeof images[0]!.mimeType, "string");
|
||||
assert.match(images[0]!.mimeType as string, /^image\//);
|
||||
});
|
||||
Reference in New Issue
Block a user