Compare commits

...

3 Commits

Author SHA1 Message Date
Diego Rodrigues de Sa e Souza
dea6bb8b6b fix(ci): publish npm from a hosted runner so provenance is accepted (#11877)
* fix(ci): publish npm from a hosted runner so provenance is accepted

The v3.8.50 staged publish failed at the upload:

  npm error code E422
  npm error 422 Unprocessable Entity - POST https://registry.npmjs.org/-/stage/package/omniroute
  Error verifying sigstore provenance bundle: Unsupported GitHub Actions runner
  environment: "self-hosted". Only "github-hosted" runners are supported when
  publishing with provenance.

3.8.49 published fine on 2026-07-30 because it predates USE_VPS_RUNNER being
turned on (2026-08-02). 3.8.50 is the first release since, so the incompatibility
had been latent for four weeks with nothing to surface it.

Neither obvious fix works on its own:

  - dropping --provenance would regress supply-chain posture; 3.8.49 carries a
    SLSA attestation and 3.8.50 must not ship without one;
  - moving the whole job to a hosted runner reintroduces the failure that made it
    self-hosted in the first place — 16 GB is not enough for build:cli's
    next-build fallback (documented on the job's runs-on).

So the work is split by what each runner is actually needed for. The self-hosted
job keeps every heavy gate — build, artifact validation, boot-smoke, the
clean-install/upgrade proof — and then packs the tarball it just proved and hands
it over. A new `stage-npm` job on ubuntu-latest downloads those exact bytes and
performs the upload, which needs no memory at all.

`npm pack --ignore-scripts` on the producing side and `--ignore-scripts` on the
publishing side both matter: prepublishOnly is `build:cli-api && build:cli &&
check:pack-artifact`, and the job already runs all three as explicit steps (the
dist/ prune is logged twice today — once at Build CLI bundle, once redundantly
inside npm stage publish). Re-running them on the small hosted runner would
rebuild bytes that were already built, validated and boot-smoked.

The DIRECT emergency fallback moved too — it published with --provenance and
would have hit the identical 422.

* chore(quality): re-baseline zizmor for the new hosted publish job

The `stage-npm` job adds 2 zizmor findings (192 -> 194), both of the same
deliberate @vN convention every workflow in this repo already follows:
unpinned-uses on actions/download-artifact@v8 and actions/setup-node@v7, plus
the cache-poisoning that setup-node@v7 already raises on the two other jobs in
this very file. SHA-pinning only the new job would break the convention.

No new class: zero template-injection, artipacked, dangerous-triggers or
excessive-permissions. The job declares contents:read + id-token:write, which is
the minimum npm provenance needs.
2026-08-28 10:26:22 -03:00
Diego Rodrigues de Sa e Souza
8e2fb04329 fix(build): drop Next trace manifests from the npm tarball (413 on publish) (#11864)
The v3.8.50 staged publish was refused by the registry:

  npm error code E413
  npm error 413 Payload Too Large - POST https://registry.npmjs.org/-/stage/package/omniroute

The tarball had reached 288.7 MB packed / 1.1 GB unpacked, against 174.5 MB /
792.3 MB for the 3.8.49 that published fine. 842 *.nft.json files accounted for
668.7 MB of that — 61% of the whole package — having doubled from the 325.0 MB
across 748 files shipped in 3.8.49.

Those are Next.js Node File Trace manifests: build-time metadata used to compute
the standalone bundle, never read while serving. Nothing under src/, open-sse/
or bin/ references them, which the new test pins.

Excluding them follows the negation pattern files[] already uses for
node_modules and test sources. Verified against an isolated package that the
glob drops page.js.nft.json while keeping page.js and other.json, so it does
not over-match.

Separately worth tracking: the compiled JS under dist/.build/next also grew 69%
between 3.8.49 and 3.8.50 (231.9 MB to 391.9 MB, +3695 files). That is not what
broke the publish and is left for its own investigation.
2026-08-28 05:29:52 -03:00
Diego Rodrigues de Sa e Souza
b7c07edad8 fix(ci): run the install-upgrade gate on disk, not on the /tmp tmpfs (#11855)
* fix(ci): run the install-upgrade gate on disk, not on the /tmp tmpfs

The v3.8.50 publish failed this gate again, and this time it said why:

  free space in /tmp: 2.9 GB
  ⚠️  only 2.9 GB free — this gate needs roughly 12 GB
  crashed: upgrade install ran out of disk space (58269 ENOSPC errors)

On the self-hosted runner `/tmp` is a **12 GB tmpfs backed by RAM**, while the
root filesystem had 66 GB free. The gate builds two ~3 GB install trees, installs
the second one over twice, and packs a 275 MB tarball — roughly 12 GB, all of it
demanded from the wrong filesystem.

This is why freeing disk never fixed it: 84 GB were freed on `/`, and none of it
ever reached the volume the gate was using. The check even measured the right
number and reported it against the wrong path, so the warning read as "the disk
is full" when the disk was fine.

- work in `<repo>/.install-upgrade/` (gitignored) instead of `os.tmpdir()`,
  overridable with `OMNIROUTE_INSTALL_UPGRADE_WORKDIR`
- the free-space log and the ENOSPC crash message now name the directory the run
  actually uses, so the next reader is sent to the filesystem that ran out

Phase A already passes on the current main: clean install healthy, version
reported correctly, 130 tables — the authentication fix and migration 163 from
#11845 both hold. Only Phase B was starved.

* docs(env): document OMNIROUTE_INSTALL_UPGRADE_WORKDIR

The workdir override introduced in this branch is a new `process.env.*` read, and
two gates caught it immediately: `issue #7793: real .env.example is in sync with
process.env.* reads in code` and `check:env-doc-sync` (Docs Sync STRICT).

Both were right — an env var that exists only in code is an env var nobody can
find. Documented in `.env.example` and `docs/reference/ENVIRONMENT.md` with the
reason it exists: the gate needs ~12 GB and must not land on a small tmpfs.
2026-08-28 00:18:02 -03:00
10 changed files with 225 additions and 37 deletions

View File

@@ -1023,6 +1023,13 @@ PROVIDER_LIMITS_SYNC_SPACING_MS=1500
# to disable the check. Used by: src/lib/db/migrationRunner.ts. Default: 50.
#OMNIROUTE_MAX_PENDING_MIGRATIONS=50
# Working directory for the check:install-upgrade release gate. It builds two ~3 GB
# install trees plus a ~275 MB tarball, so it needs roughly 12 GB — more than the
# 12 GB RAM-backed tmpfs that /tmp is on the self-hosted runner, where it exhausted
# the tmpfs and npm silently truncated the package. Defaults to <repo>/.install-upgrade
# on real disk. Used by: scripts/check/check-install-upgrade.mjs. Default: <repo>/.install-upgrade.
#OMNIROUTE_INSTALL_UPGRADE_WORKDIR=/var/tmp/omniroute-install-upgrade
# Trust user-managed RTK project filter rules without strict signature checks.
# Used by: open-sse/services/compression/engines/rtk/filterLoader.ts. Default: 0.
#OMNIROUTE_RTK_TRUST_PROJECT_FILTERS=0

View File

@@ -63,10 +63,14 @@ jobs:
# This job never runs on `pull_request`, so the fork-safety clause is always true here;
# it is kept verbatim so the expression stays greppable against ci.yml.
runs-on: ${{ (vars.USE_VPS_RUNNER == 'true' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository)) && fromJSON('["self-hosted","omni-release"]') || 'ubuntu-latest' }}
outputs:
version: ${{ steps.resolve.outputs.version }}
tag: ${{ steps.resolve.outputs.tag }}
skip: ${{ steps.resolve.outputs.skip }}
permissions:
actions: read # find + download the CI run's next-build artifact for this SHA
contents: write # gh release upload (attach SBOM to the GitHub Release)
id-token: write # npm provenance
id-token: write # npm provenance (GitHub Packages step)
packages: write # publish to npm.pkg.github.com
steps:
- name: Checkout
@@ -314,41 +318,33 @@ jobs:
fi
npm --version
- name: Publish to npm (staged — owner approves with 2FA)
if: steps.resolve.outputs.skip != 'true' && (github.event_name != 'workflow_dispatch' || inputs.publish_mode != 'direct')
# The registry upload itself moved to the `stage-npm` job below: npm REFUSES
# `--provenance` from a self-hosted runner (422 "Unsupported GitHub Actions
# runner environment"), and the heavy verification above cannot move to a
# hosted one (16 GB is not enough for build:cli's next-build fallback — see
# this job's runs-on comment). So this job proves the bytes and hands them
# over; a tiny hosted job does the upload.
- name: Pack the verified tarball for the upload job
if: steps.resolve.outputs.skip != 'true'
env:
VERSION: ${{ steps.resolve.outputs.version }}
TAG: ${{ steps.resolve.outputs.tag }}
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
run: |
set -euo pipefail
# Always pass --tag explicitly. Defense in depth: even if VERSION is
# accidentally an older release, the historic tag will NOT claim `@latest`.
npm stage publish --provenance --access public --tag "$TAG"
{
echo "## 📦 omniroute@$VERSION STAGED (not yet installable)"
echo ""
echo "The exact bytes are parked on the registry. To release them:"
echo '```'
echo "npm stage list omniroute # find the stage id"
echo "npm stage approve <id> # owner 2FA — THE publish"
echo '```'
echo "To verify the staged bytes first: npm stage download <id> → run"
echo "scripts/check/check-pack-boot.mjs against them (see RELEASE_CHECKLIST)."
echo "To discard: npm stage reject <id>."
} >> "$GITHUB_STEP_SUMMARY"
echo "✅ Staged omniroute@$VERSION (dist-tag=$TAG) — awaiting owner 'npm stage approve'"
# --ignore-scripts: prepublishOnly would re-run build:cli-api && build:cli,
# rebuilding bytes this job has already built, validated and boot-smoked.
npm pack --ignore-scripts
TARBALL="omniroute-${VERSION}.tgz"
test -f "$TARBALL" || { echo "expected $TARBALL to exist after npm pack" >&2; ls -la ./*.tgz || true; exit 1; }
echo "packed $TARBALL ($(du -h "$TARBALL" | cut -f1))"
- name: Publish to npm (DIRECT — emergency fallback)
if: steps.resolve.outputs.skip != 'true' && github.event_name == 'workflow_dispatch' && inputs.publish_mode == 'direct'
env:
VERSION: ${{ steps.resolve.outputs.version }}
TAG: ${{ steps.resolve.outputs.tag }}
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
run: |
set -euo pipefail
npm publish --provenance --access public --tag "$TAG"
echo "✅ Published omniroute@$VERSION (dist-tag=$TAG) [DIRECT mode]"
- name: Hand the tarball to the hosted publish job
if: steps.resolve.outputs.skip != 'true'
uses: actions/upload-artifact@v7
with:
name: npm-tarball
path: omniroute-${{ steps.resolve.outputs.version }}.tgz
retention-days: 1
if-no-files-found: error
- name: Publish to GitHub Packages
if: steps.resolve.outputs.skip != 'true'
@@ -365,6 +361,91 @@ jobs:
|| echo "⚠️ omniroute@${VERSION} might already be published on GitHub Packages."
echo "✅ Action finished for GitHub Packages"
# npm REFUSES `--provenance` from a self-hosted runner:
# 422 Unprocessable Entity - Error verifying sigstore provenance bundle:
# Unsupported GitHub Actions runner environment: "self-hosted".
# Only "github-hosted" runners are supported when publishing with provenance.
# v3.8.49 published fine because it predates USE_VPS_RUNNER being turned on
# (2026-08-02); v3.8.50 was the first release after it, so this had been latent
# for four weeks. Dropping --provenance was not an option: 3.8.49 carries a
# SLSA attestation and 3.8.50 must not regress that.
# The `publish` job cannot simply move to a hosted runner either — 16 GB is not
# enough for build:cli's next-build fallback. So it keeps proving the bytes and
# this job, which needs no memory at all, performs the upload.
stage-npm:
needs: publish
if: needs.publish.outputs.skip != 'true'
runs-on: ubuntu-latest
permissions:
contents: read
id-token: write # npm provenance — the whole reason this job is separate
steps:
- name: Download the tarball the publish job proved
uses: actions/download-artifact@v8
with:
name: npm-tarball
path: .
- name: Setup Node.js
uses: actions/setup-node@v7
with:
node-version: ${{ env.NPM_PUBLISH_NODE_VERSION }}
registry-url: https://registry.npmjs.org
- name: Ensure npm supports staged publishing
run: |
set -euo pipefail
CUR=$(npm --version)
if ! node -e "const [a,b]='$(npm --version)'.split('.').map(Number); process.exit(a>11||(a===11&&b>=15)?0:1)"; then
# Pinned exact version (supply-chain: never float @latest in a publish
# job); bump deliberately when a newer npm is required.
echo "npm $CUR < 11.15 — installing pinned npm 11.15.0 for staged publishing"
npm install -g --ignore-scripts npm@11.15.0
fi
npm --version
- name: Publish to npm (staged — owner approves with 2FA)
if: github.event_name != 'workflow_dispatch' || inputs.publish_mode != 'direct'
env:
VERSION: ${{ needs.publish.outputs.version }}
TAG: ${{ needs.publish.outputs.tag }}
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
run: |
set -euo pipefail
TARBALL="omniroute-${VERSION}.tgz"
test -f "$TARBALL" || { echo "tarball $TARBALL did not arrive from the publish job" >&2; ls -la; exit 1; }
# Always pass --tag explicitly. Defense in depth: even if VERSION is
# accidentally an older release, the historic tag will NOT claim `@latest`.
# --ignore-scripts: publishing a built tarball must never re-run
# prepublishOnly (build:cli-api && build:cli) on this small runner.
npm stage publish "$TARBALL" --provenance --access public --tag "$TAG" --ignore-scripts
{
echo "## 📦 omniroute@$VERSION STAGED (not yet installable)"
echo ""
echo "The exact bytes are parked on the registry. To release them:"
echo '```'
echo "npm stage list omniroute # find the stage id"
echo "npm stage approve <id> # owner 2FA — THE publish"
echo '```'
echo "To verify the staged bytes first: npm stage download <id> → run"
echo "scripts/check/check-pack-boot.mjs against them (see RELEASE_CHECKLIST)."
echo "To discard: npm stage reject <id>."
} >> "$GITHUB_STEP_SUMMARY"
echo "✅ Staged omniroute@$VERSION (dist-tag=$TAG) — awaiting owner 'npm stage approve'"
- name: Publish to npm (DIRECT — emergency fallback)
if: github.event_name == 'workflow_dispatch' && inputs.publish_mode == 'direct'
env:
VERSION: ${{ needs.publish.outputs.version }}
TAG: ${{ needs.publish.outputs.tag }}
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
run: |
set -euo pipefail
TARBALL="omniroute-${VERSION}.tgz"
test -f "$TARBALL" || { echo "tarball $TARBALL did not arrive from the publish job" >&2; ls -la; exit 1; }
npm publish "$TARBALL" --provenance --access public --tag "$TAG" --ignore-scripts
echo "✅ Published omniroute@$VERSION (dist-tag=$TAG) [DIRECT mode]"
publish-opencode-plugin:
runs-on: ubuntu-latest
permissions:

3
.gitignore vendored
View File

@@ -293,3 +293,6 @@ docker-compose.yml.bak
# Ad-hoc test sandboxes (never tracked — may contain local DBs)
/.sandbox/
.aider*
# check:install-upgrade work trees (~12 GB, disposable)
/.install-upgrade/

View File

@@ -0,0 +1,4 @@
- Excluded Next.js Node File Trace manifests (`*.nft.json`) from the published npm
tarball. They are build-time metadata and are never read while serving, but had
grown to 668.7 MB — 61% of the package — which pushed the upload past the
registry limit and made `npm publish` fail with `413 Payload Too Large`.

View File

@@ -0,0 +1,4 @@
- Split the npm registry upload into its own GitHub-hosted job. npm refuses
`--provenance` from a self-hosted runner (`422 ... Only "github-hosted" runners
are supported`), which blocked the v3.8.50 publish; the heavy verification
cannot move to a hosted runner, so it now hands the proven tarball over instead.

View File

@@ -169,7 +169,7 @@
"dedicatedGate": true
},
"zizmorFindings": {
"value": 192,
"value": 194,
"_rebaseline_2026_08_20_radar_export_workflow": "190 -> 192 (+2). Workflow novo `.github/workflows/radar-export.yml` (passo 10 do go-live do Radar: publica o export estável do catálogo como asset de release para o servidor privado baixar via RADAR_EXPORT_URL). Os +2 são unpinned-uses @vN: actions/checkout@v7 + actions/setup-node@v7 — a MESMA convenção deliberada de todos os workflows (ver _scanner_harden_workflows_2026_06_16); fixar por SHA só este violaria a convenção. O findings artipacked do checkout foi CORRIGIDO com `persist-credentials: false` (o job publica via GH_TOKEN em `gh release`, não usa a credencial do checkout). Nenhuma classe nova de template-injection / cache-poisoning / dangerous-triggers. Medido local com zizmor 1.25.2 via `node scripts/check/check-workflows.mjs --ratchet` = 191; +1 do delta conhecido do runner (ver _rebaseline_2026_07_28_ci_runner_delta: o runner enxerga 1 unpinned-uses @vN a mais que o devbox no mesmo commit; a baseline segue o runner) => 192.",
"_rebaseline_2026_07_20_aliasresolver_hook_split_7808": "175 -> 176 (+1). Companion to PR #7808 (CodeQL js/incomplete-url-substring-sanitization fix in bin/aliasResolver.mjs). The +1 is NOT caused by this PR's code changes (bin/* is not a workflow file) — it is a pre-existing drift that surfaced because the ratchet gate runs on this PR's CI: the zizmor scanner version on the GitHub runner gained a new rule (or extended an existing one) since the v3.8.49 baseline was seeded on 2026-07-17. Breakdown: the new finding is an unpinned-uses @vN class item on one of the existing workflows (same deliberate convention as _scanner_harden_workflows_2026_06_16 — @vN is intentional, SHA-pinning only this one would violate the convention). No new template-injection/artipacked/cache-poisoning/dangerous-triggers classes introduced. Measured by the Quality Gates (Extended) job on run 29713001401 = 176, baseline was 175. Note: by the time this landed on release/v3.8.49, the baseline was already at 176 via _rebaseline_2026_07_17_combo_recovery_hints — this entry is kept as historical record; no further bump applied.",
"_rebaseline_2026_07_17_v3849_release": "169 -> 175 (+6). Cycle workflow drift (v3.8.48/v3.8.49): npm-publish.yml (new, WS1.3 #7092), electron-release.yml, nightly-compat.yml, nightly-release-green.yml, CI restructures (#7501 full-history base fetch, #7355 main-green, #7202 merge-queue gates, Trunk/Codecov). Breakdown vs v3.8.47: +3 unpinned-uses (@vN convention, deliberate per _scanner_harden_workflows_2026_06_16), +2 cache-poisoning (artifact upload/cache in the OWN electron-release/npm-publish RELEASE workflows -- operator-controlled, not fork-PR exploitable), +1 excessive-permissions (nightly-compat.yml permissions:issues). No new template-injection/artipacked/dangerous-triggers. Measured with zizmor 1.25.2 via `node scripts/check/check-workflows.mjs --ratchet` = 175 on da3a0be69.",
@@ -180,7 +180,8 @@
"_rebaseline_2026_06_23_v3834_release": "152 -> 155 (+3). The 3 new unpinned-uses are in .github/workflows/nightly-release-green.yml (added by #4622 this cycle): actions/checkout@v7, actions/setup-node@v6, actions/upload-artifact@v4 — the SAME deliberate @vN convention as ci.yml's own checkout@v7/setup-node@v6 and every other workflow (see _scanner_harden_workflows_2026_06_16 + _zizmor_rebaseline_2026_06_20_ci_build_artifact_reuse). SHA-pinning only this workflow would violate the convention. The workflow-lint ratchet does NOT run on PR->release fast-gates, so it surfaced only on the release PR; measured locally via `npm run check:workflows -- --ratchet` = 155. No new template-injection/artipacked/cache-poisoning.",
"_rebaseline_2026_07_13_v3847_release_preflight": "159 -> 169 (+10). Findings from cycle-merged workflow changes: #6716 (PR gate restructure), #6781 (unit fast-path shard 2->4), #6788 (TIA tsx loader split), #6881 (electron-updater latest.yml manifests in release assets) — same deliberate @vN unpinned-uses convention as prior rebaselines; no new template-injection/artipacked/cache-poisoning classes. Measured via `npm run check:workflows -- --ratchet` = 169 on the v3.8.47 release pre-flight.",
"_rebaseline_2026_07_28_v3849_release_preflight": "176 -> 189 (+13). Pre-flight de fechamento da v3.8.49 (934 commits no ciclo). Deriva de workflow: 1 workflow novo (build-rinseaid-image.yml) mais os bumps de action do Dependabot ao longo do ciclo — todos da MESMA classe unpinned-uses @vN, convenção deliberada do repo (ver _scanner_harden_workflows_2026_06_16); fixar por SHA só estes violaria a convenção. Nenhuma classe nova de template-injection / artipacked / cache-poisoning / dangerous-triggers. Nesta mesma passada foram CORRIGIDAS 3 diretivas shellcheck malformadas (SC1125: `# shellcheck disable=SC2086 — texto`, em que o travessão invalida o par key=value) em ci.yml e nightly-release-green.yml. Medido com zizmor 1.25.2 via `npm run check:workflows -- --ratchet` = 189.",
"_rebaseline_2026_07_28_ci_runner_delta": "189 -> 190 (+1). Medido 189 no devbox e 190 no runner do GitHub no MESMO commit (run 30396592013, job Quality Gates (Extended)) — mesma classe já registrada em _rebaseline_2026_07_20_aliasresolver_hook_split_7808: a versão do zizmor no runner enxerga uma finding a mais que a local, sempre da classe unpinned-uses @vN. O valor do runner é o que o gate compara, então a baseline segue o runner."
"_rebaseline_2026_07_28_ci_runner_delta": "189 -> 190 (+1). Medido 189 no devbox e 190 no runner do GitHub no MESMO commit (run 30396592013, job Quality Gates (Extended)) — mesma classe já registrada em _rebaseline_2026_07_20_aliasresolver_hook_split_7808: a versão do zizmor no runner enxerga uma finding a mais que a local, sempre da classe unpinned-uses @vN. O valor do runner é o que o gate compara, então a baseline segue o runner.",
"_rebaseline_2026_08_28_npm_publish_hosted_stage_job": "192 -> 194 (+2). Job novo `stage-npm` em .github/workflows/npm-publish.yml: o npm RECUSA `--provenance` vindo de runner self-hosted (422 \"Unsupported GitHub Actions runner environment\"), e o job `publish` nao pode migrar para runner hospedado porque 16 GB nao bastam para o fallback next-build do build:cli (documentado no proprio runs-on). A separacao foi a unica saida que preserva a atestacao SLSA que a 3.8.49 ja tem. Os +2 sao da MESMA convencao deliberada de todos os workflows (ver _scanner_harden_workflows_2026_06_16): unpinned-uses @vN em actions/download-artifact@v8 + actions/setup-node@v7, mais o cache-poisoning que o proprio setup-node@v7 ja gera nos outros 2 jobs deste MESMO arquivo (linhas 85 e 463) e que ja esta na baseline. Fixar por SHA so este job violaria a convencao. Nenhuma classe nova: zero template-injection / artipacked / dangerous-triggers / excessive-permissions — o job declara apenas contents:read + id-token:write, que e o minimo para a proveniencia. Medido pelo job Quality Gates (Extended) no run 33162... da PR #11877 = 194."
},
"vulnCount": {
"value": 22,

View File

@@ -103,6 +103,7 @@ OmniRoute uses **SQLite** (via `better-sqlite3`) for all persistence. These vari
| `OMNIROUTE_MIGRATIONS_DIR` | _(auto-detect)_ | `src/lib/db/migrationRunner.ts` | Override the directory that the migration runner scans. Useful when shipping bundled migrations in custom builds. |
| `OMNIROUTE_EXTRA_MIGRATIONS_DIRS` | _(unset)_ | `src/lib/db/migrationRunner/extraDirs.ts` | Additional migration directories as `namespace=dir` entries separated by the platform path delimiter (e.g. `ee=/opt/app/enterprise/db/migrations`). Files found there are recorded as `<namespace>-<number>`, so a distribution shipping its own migrations never collides with the upstream numeric slots. A malformed entry, an invalid namespace or a missing directory throws at startup instead of silently skipping the schema. |
| `OMNIROUTE_MAX_PENDING_MIGRATIONS` | `50` | `src/lib/db/migrationRunner.ts` | Mass-pending-migrations safety threshold (#3416). Startup aborts if more than this many migrations are pending on an existing DB (guards against a wiped tracking table). Raise it to restore an older backup; set to `0` to disable the check. |
| `OMNIROUTE_INSTALL_UPGRADE_WORKDIR` | _(`<repo>/.install-upgrade`)_ | `scripts/check/check-install-upgrade.mjs` | Working directory for the `check:install-upgrade` release gate. It needs roughly 12 GB (two ~3 GB install trees plus the tarball), so it must not run on a small tmpfs — on the self-hosted runner `/tmp` is a 12 GB RAM-backed tmpfs and the gate exhausted it, truncating the package. |
| `OMNIROUTE_SPEND_FLUSH_INTERVAL_MS` | _(default in code)_ | `src/lib/spend/batchWriter.ts` | Flush interval (ms) for the batched spend/cost writer. Lower values reduce write coalescing; higher values reduce DB contention. |
| `OMNIROUTE_SPEND_MAX_BUFFER_SIZE` | _(default in code)_ | `src/lib/spend/batchWriter.ts` | Max buffered spend entries before a forced flush. Raise on high-QPS deployments; lower when bounded memory matters more. |
| `OMNIROUTE_PROXY_FETCH_DEBUG` | _(unset)_ | `open-sse/utils/proxyFetch.ts` | Set to `"true"` to emit `[ProxyFetch]` debug logs on the Vercel relay path. Off by default to avoid leaking routing hints. |

View File

@@ -52,7 +52,8 @@
"!**/*.test.js",
"!**/*.test.mjs",
"!**/*.spec.ts",
"!**/*.spec.tsx"
"!**/*.spec.tsx",
"!**/*.nft.json"
],
"workspaces": [
"open-sse",

View File

@@ -302,7 +302,7 @@ export function assertNoDiskExhaustion(output, label) {
throw new Error(
`${label}: the install ran out of disk space (${count} ENOSPC error(s) from npm). ` +
`The package tree is truncated, so anything measured from it — boot, schema, ` +
`migrations — is meaningless. Free space in ${os.tmpdir()} (each install tree is ` +
`migrations — is meaningless. Free space in ${workDirForMessages} (each install tree is ` +
`~3 GB) and re-run. This is an environment failure, NOT a schema divergence.`
);
}
@@ -318,6 +318,11 @@ function freeBytes(dir) {
const GB = 1024 ** 3;
// Set once the work directory exists, so the ENOSPC message names the filesystem that
// actually ran out — pointing at /tmp when the gate works elsewhere sends the reader to
// free space on the wrong volume (which is what happened during the v3.8.50 publish).
let workDirForMessages = os.tmpdir();
function resolvePreviousVersion(current, explicit) {
if (explicit) return explicit;
const out = execFileSync("npm", ["view", "omniroute", "dist-tags.latest"], { encoding: "utf8" });
@@ -348,7 +353,16 @@ async function main() {
}
const version = JSON.parse(fs.readFileSync(path.join(ROOT, "package.json"), "utf8")).version;
const allowlist = loadAllowlist(ROOT);
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-install-upgrade-"));
// NOT os.tmpdir(): on the self-hosted runner /tmp is a 12 GB tmpfs backed by RAM, while
// the root filesystem has ~66 GB free. This gate needs ~12 GB, so it exhausted the tmpfs
// and npm truncated the package — 58269 ENOSPC errors on the v3.8.50 publish, which the
// previous code could only report as a crash. Freeing disk did not help because the disk
// was never the constraint. Work on real disk beside the repo instead.
const workRoot =
process.env.OMNIROUTE_INSTALL_UPGRADE_WORKDIR || path.join(ROOT, ".install-upgrade");
fs.mkdirSync(workRoot, { recursive: true });
const tmp = fs.mkdtempSync(path.join(workRoot, "omniroute-install-upgrade-"));
workDirForMessages = tmp;
const failures = [];
const warnings = [];
@@ -374,7 +388,7 @@ async function main() {
// exited 0, and every later measurement was taken from a broken tree.
const availableBytes = freeBytes(tmp);
if (availableBytes !== null) {
log(`free space in ${os.tmpdir()}: ${(availableBytes / GB).toFixed(1)} GB`);
log(`free space in ${tmp}: ${(availableBytes / GB).toFixed(1)} GB`);
if (availableBytes < 12 * GB) {
warn(
`only ${(availableBytes / GB).toFixed(1)} GB free — this gate needs roughly 12 GB ` +

View File

@@ -0,0 +1,72 @@
import test from "node:test";
import assert from "node:assert/strict";
import { readFileSync, readdirSync } from "node:fs";
import type { Dirent } from "node:fs";
import { join } from "node:path";
/**
* v3.8.50 was refused by the registry with `413 Payload Too Large` on
* `POST /-/stage/package/omniroute`: the tarball had reached 288.7 MB packed
* (1.1 GB unpacked), against 174.5 MB for the 3.8.49 that published fine.
*
* 668.7 MB of that — 61% of the whole package — was 842 `*.nft.json` files.
* Those are Next.js Node File Trace manifests: build-time metadata used to
* COMPUTE the standalone bundle, never read while serving. They had doubled
* since 3.8.49 (325.0 MB across 748 files), which is what tipped the payload
* over the limit.
*
* The guard is the `files[]` negation, so a future entry that re-widens the
* glob (or a rewrite of the array) cannot silently put them back.
*/
const pkg = JSON.parse(readFileSync(join(import.meta.dirname, "../../package.json"), "utf8")) as {
files?: string[];
};
test("package.json files[] excludes Next's .nft.json trace manifests", () => {
const files = pkg.files ?? [];
assert.ok(files.length > 0, "package.json must declare files[]");
assert.ok(
files.includes("!**/*.nft.json"),
"files[] must negate **/*.nft.json — they are build metadata and were 61% of the 3.8.50 payload"
);
});
test("the negation sits after the positive dist/ entry it has to override", () => {
// npm applies files[] in order: a negation listed BEFORE the directory that
// pulls the files in is a no-op. Positive anchor, so this test cannot pass
// just because both strings happen to be present somewhere.
const files = pkg.files ?? [];
const dist = files.indexOf("dist/");
const negation = files.indexOf("!**/*.nft.json");
assert.notEqual(dist, -1, "dist/ must still be published");
assert.ok(negation > dist, "the .nft.json negation must come after dist/");
});
test("no source module reads a .nft.json at runtime", () => {
// If this ever stops holding, the exclusion above becomes a runtime break
// rather than a size win — which is exactly the assumption worth pinning.
const roots = ["src", "open-sse", "bin"];
const hits: string[] = [];
for (const root of roots) {
const dir = join(import.meta.dirname, "../..", root);
const stack = [dir];
while (stack.length > 0) {
const current = stack.pop() as string;
let entries: Dirent[];
try {
entries = readdirSync(current, { withFileTypes: true });
} catch {
continue;
}
for (const entry of entries) {
const full = join(current, entry.name);
if (entry.isDirectory()) {
if (entry.name !== "node_modules") stack.push(full);
} else if (/\.(ts|tsx|mjs|js)$/.test(entry.name)) {
if (readFileSync(full, "utf8").includes(".nft.json")) hits.push(full);
}
}
}
}
assert.deepEqual(hits, [], `nothing may depend on .nft.json at runtime: ${hits.join(", ")}`);
});