From 7eca04fd12d73ed7bb353dbb9868992d4795f2b8 Mon Sep 17 00:00:00 2001 From: Diego Rodrigues de Sa e Souza Date: Thu, 30 Jul 2026 09:07:57 -0300 Subject: [PATCH] feat(ci): gate the publish on clean-install AND upgrade-over-previous (#8953) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(ci): gate the publish on clean-install AND upgrade-over-previous `check:pack-boot` proves a fresh install boots. It does not prove the path that actually broke us: installing over an existing version, where ~110 SQLite migrations run against a populated database. v3.8.48 shipped as a hotfix because the published 3.8.47 crashed on boot, and the v3.8.49 upgrade path was only ever exercised end-to-end by hand — on VPS .16, against a real 3.8.48 install with a 165 MB database, AFTER publishing. That is backwards. New gate (`scripts/check/check-install-upgrade.mjs`), wired into npm-publish.yml as step 12, BEFORE `npm stage publish` — so a broken upgrade never reaches the registry and a staged package that is never approved simply expires, with no `npm deprecate` needed: - Phase A: fresh prefix + fresh DATA_DIR, install the packed tarball, boot, health. - Phase B: fresh prefix + fresh DATA_DIR, install the PREVIOUS published version, boot it (creates + migrates the DB), stop, install the tarball over the SAME prefix, boot against the SAME DATA_DIR. Asserts no table present before the upgrade was dropped. - Schema convergence, and its DIRECTION is the whole point: fresh − upgraded ≠ ∅ → FAIL. Structure a clean install creates but an upgrade does not means every existing user is missing it. Not allowlistable. upgraded − fresh ≠ ∅ → residue; fails only when NEW (allowlist carries the known ones). A naive symmetric check would either block every release on harmless residue or, if relaxed, let the dangerous direction through. Measured on VPS .16 (2026-07-30): a real 3.8.48 install upgraded to 3.8.49 ended with 117 tables against 116 for a clean 3.8.49 install — the extra being `cache_metrics`, recorded in config/quality/install-upgrade-allowlist.json with the measurement. Both installs healthy, zero `no such table` in 150 log lines. `evaluateConvergence` is exported and pure so the asymmetry is testable without packing, installing or booting anything (same reason check-test-masking exports its helpers): tests/unit/check-install-upgrade-convergence.test.ts, 8 cases, ~6ms. A previous version that fails to boot degrades to a warning — a historically bad publish must not block the current one. Uses node:sqlite (Node 24, already the publish job's runtime): no new dependency. * fix(ci): require the reused next-build artifact to come from this repository CodeQL raised actions/artifact-poisoning/critical on the `next-build` fast path this PR builds on (#8941). The finding is real and it sits on the path that produces the published npm tarball. The step picks a CI run by querying the runs API for `head_sha` and filtering on `name == "CI" and conclusion == "success"`. That query also returns `pull_request` runs from FORKS: they execute in this repository's context and upload their own `next-build`, built from fork-controlled source. Measured today, 57 runs in this repo have a `head_repository` other than the repo itself. So the selection trusted bytes by coincidence of commit SHA — anything that made a fork's head commit coincide with the publish commit could put attacker-built bytes on npm. Adds `and .head_repository.full_name == env.REPO` to the selection. Provenance is now explicit; `head_sha` still carries tree-equality. Verified against the live API using the expression extracted from the workflow itself — the same single run (30518663668) is selected either way for the current tip, so the fast path keeps working while every fork run is excluded. Not a dismissal (hard rule #14) — the clause removes the flagged trust. node --import tsx/esm --test tests/unit/npm-publish-artifact-provenance.test.ts # 3 pass, 0 fail (base: 2 pass, 1 fail) --------- Co-authored-by: diegosouzapw --- .github/workflows/npm-publish.yml | 29 +- config/quality/install-upgrade-allowlist.json | 6 + package.json | 1 + scripts/check/check-install-upgrade.mjs | 342 ++++++++++++++++++ .../check-install-upgrade-convergence.test.ts | 95 +++++ .../npm-publish-artifact-provenance.test.ts | 90 +++++ 6 files changed, 562 insertions(+), 1 deletion(-) create mode 100644 config/quality/install-upgrade-allowlist.json create mode 100644 scripts/check/check-install-upgrade.mjs create mode 100644 tests/unit/check-install-upgrade-convergence.test.ts create mode 100644 tests/unit/npm-publish-artifact-provenance.test.ts diff --git a/.github/workflows/npm-publish.yml b/.github/workflows/npm-publish.yml index f82ff1b669..ea9cad488d 100644 --- a/.github/workflows/npm-publish.yml +++ b/.github/workflows/npm-publish.yml @@ -159,6 +159,17 @@ jobs: # `head_sha` is the tree-equality guarantee: same commit, same tree. # Best-effort by design (retention is 1 day): every miss falls through to the build # step below, which is why the dynamic runner above matters as the backstop. + # + # The `head_repository.full_name == env.REPO` clause is a supply-chain guard, not a + # filter refinement. This artifact becomes the published npm tarball. `pull_request` + # runs from forks execute in THIS repository's context and upload their own + # `next-build` built from fork-controlled source, and the runs API returns them for a + # matching `head_sha` — 57 such runs exist in this repo today. Without the clause, + # anything that made a fork's head commit coincide with the publish commit could put + # attacker-built bytes on npm. Requiring the run to originate from this repository + # excludes every fork run while keeping the fast path intact (verified: the same + # single run is selected either way for the current tip). + # CodeQL: actions/artifact-poisoning/critical. - name: Reuse CI's next-build artifact (skips the heavy rebuild) if: steps.resolve.outputs.skip != 'true' continue-on-error: true @@ -169,7 +180,11 @@ jobs: run: | set -uo pipefail RUN=$(gh api "repos/$REPO/actions/runs?head_sha=$HEAD_SHA&per_page=100" \ - --jq '[.workflow_runs[] | select(.name == "CI" and .conclusion == "success")] | .[0].id // empty') || RUN="" + --jq '[.workflow_runs[] + | select(.name == "CI" + and .conclusion == "success" + and .head_repository.full_name == env.REPO)] + | .[0].id // empty') || RUN="" if [ -z "$RUN" ]; then echo "::notice::no successful CI run for $HEAD_SHA — falling back to a full build" exit 0 @@ -223,6 +238,18 @@ jobs: if: steps.resolve.outputs.skip != 'true' run: npm run check:pack-boot + # The boot-smoke above proves a CLEAN install boots. It does not prove the path that + # actually broke us: installing over an existing version, where ~110 SQLite migrations + # run against a populated database. v3.8.48 shipped as a hotfix because the published + # 3.8.47 crashed on boot, and the v3.8.49 upgrade path was first exercised end-to-end + # by hand on a real 3.8.48 box (VPS .16) — after publishing, which is exactly backwards. + # Runs BEFORE `npm stage publish` so a broken upgrade never reaches the registry at all; + # a staged package that is never approved simply expires, with no `npm deprecate` needed. + - name: Prove clean-install AND upgrade-over-previous both boot + if: steps.resolve.outputs.skip != 'true' + timeout-minutes: 30 + run: npm run check:install-upgrade + # WS1.3 (D2, v3.8.49 plan): STAGED publishing by default — `npm stage publish` # parks the exact bytes on the registry WITHOUT making them installable; the # owner then verifies and approves with 2FA (`npm stage approve`), moving the diff --git a/config/quality/install-upgrade-allowlist.json b/config/quality/install-upgrade-allowlist.json new file mode 100644 index 0000000000..4e6f3c6891 --- /dev/null +++ b/config/quality/install-upgrade-allowlist.json @@ -0,0 +1,6 @@ +{ + "_doc": "Tables that exist ONLY in databases upgraded from an older version — residue whose CREATE left the migration set in some past cycle but survives where it already existed. Harmless (nothing references them), but recorded here so check-install-upgrade.mjs can still fail on a NEW divergence. The opposite direction (a table a clean install creates but an upgrade does not) is NEVER allowlisted: it means every existing user is missing structure the code expects.", + "residualTables": { + "cache_metrics": "Measured 2026-07-30 on a real 3.8.48 install upgraded to 3.8.49 (VPS .16, 165 MB database, 114 → 117 tables). Present in upgraded databases, absent from clean installs. No code path referenced it during the upgrade (zero `no such table` in 150 log lines, both installs healthy). Left in place rather than dropped: a DROP migration on a table we cannot prove is unused everywhere is the riskier change. Revisit when the cache subsystem is next touched." + } +} diff --git a/package.json b/package.json index 100bb0519d..cf77746b07 100644 --- a/package.json +++ b/package.json @@ -143,6 +143,7 @@ "check:node-runtime": "node --import tsx scripts/check/check-supported-node-runtime.ts", "check:pack-artifact": "node --import tsx scripts/build/validate-pack-artifact.ts", "check:pack-boot": "node scripts/check/check-pack-boot.mjs", + "check:install-upgrade": "node scripts/check/check-install-upgrade.mjs", "check:pack-policy": "node --import tsx scripts/build/validate-pack-artifact.ts --policy-only", "check:cli-i18n": "node scripts/check/check-cli-i18n.mjs", "check:openapi-coverage": "node scripts/check/check-openapi-coverage.mjs", diff --git a/scripts/check/check-install-upgrade.mjs b/scripts/check/check-install-upgrade.mjs new file mode 100644 index 0000000000..87253c611d --- /dev/null +++ b/scripts/check/check-install-upgrade.mjs @@ -0,0 +1,342 @@ +#!/usr/bin/env node +/** + * check-install-upgrade — proves the two install paths a real user takes, BEFORE publishing. + * + * `check:pack-boot` already proves a fresh install boots. It does NOT prove the path that + * actually broke us: installing the new version OVER an existing one, where ~110 SQLite + * migrations run against a populated database. v3.8.48 shipped as a hotfix precisely because + * the published 3.8.47 crashed on boot, and a v3.8.49 manual test on a real 3.8.48 box was + * what first exercised the upgrade path end to end. + * + * Phase A — clean install: fresh prefix + fresh DATA_DIR, install the packed tarball, boot. + * Phase B — upgrade install: fresh prefix + fresh DATA_DIR, install the PREVIOUS published + * version, boot it (creates + migrates the DB), stop, install the + * packed tarball over the SAME prefix, boot against the SAME DATA_DIR. + * + * Schema convergence is the third assertion, and its DIRECTION is what matters: + * + * fresh − upgraded ≠ ∅ → FAIL. A table a clean install creates but an upgrade does not + * means every existing user is missing structure the code expects. + * This is the failure mode that only ever bites upgraders. + * upgraded − fresh ≠ ∅ → WARN. Residue: a table whose CREATE left the migration set in + * some past cycle but survives in databases that already had it. + * Harmless, but it means the two paths do not converge — allowlist + * it explicitly so a NEW divergence is still visible. + * + * Usage: + * node scripts/check/check-install-upgrade.mjs [--from ] [--skip-upgrade] + * + * `--from` pins the previous version (default: the current `latest` dist-tag on npm). + * Requires `npm run build:cli` first — this is a --with-build gate, like check:pack-boot. + */ + +import { execFileSync, spawn } from "node:child_process"; +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import { DatabaseSync } from "node:sqlite"; + +const BOOT_DEADLINE_MS = 180_000; +const POLL_INTERVAL_MS = 2_000; +const ALLOWLIST_PATH = "config/quality/install-upgrade-allowlist.json"; + +const log = (msg) => console.log(`[install-upgrade] ${msg}`); +const warn = (msg) => console.log(`[install-upgrade] ⚠️ ${msg}`); + +function pickTarball(packJson) { + const filename = JSON.parse(packJson)?.[0]?.filename; + if (!filename) throw new Error("npm pack --json returned no filename"); + return filename; +} + +/** Free-ish port per phase so a leaked child from a previous run cannot collide. */ +function pickPort(offset) { + return 21000 + offset + (process.pid % 500); +} + +function loadAllowlist(root) { + const file = path.join(root, ALLOWLIST_PATH); + if (!fs.existsSync(file)) return { residualTables: {} }; + return JSON.parse(fs.readFileSync(file, "utf8")); +} + +/** + * Pure verdict on schema convergence — exported so the asymmetry can be tested without + * building, packing and booting anything (the same reason check-test-masking exports its + * helpers: reproducing the deterministic part must not cost a full gate run). + * + * The two directions are NOT symmetric: + * onlyFresh → always a failure. Upgraders would be missing structure. + * onlyUpgraded → residue. Fails only when not recorded in the allowlist. + */ +export function evaluateConvergence({ freshTables, upgradedTables, residualAllowlist = {} }) { + const fresh = freshTables instanceof Set ? freshTables : new Set(freshTables ?? []); + const upgraded = upgradedTables instanceof Set ? upgradedTables : new Set(upgradedTables ?? []); + const onlyFresh = [...fresh].filter((t) => !upgraded.has(t)).sort(); + const onlyUpgraded = [...upgraded].filter((t) => !fresh.has(t)).sort(); + const unknownResidue = onlyUpgraded.filter((t) => !(t in residualAllowlist)); + const failures = []; + if (onlyFresh.length) { + failures.push( + `schema divergence — tables a CLEAN install creates but an UPGRADE does not: ${onlyFresh.join(", ")}. ` + + "Every existing user would be missing these; add the migration." + ); + } + if (unknownResidue.length) { + failures.push( + `NEW residual table(s) not in ${ALLOWLIST_PATH}: ${unknownResidue.join(", ")}. ` + + "Either drop them in a migration or record them with a justification." + ); + } + return { ok: failures.length === 0, failures, onlyFresh, onlyUpgraded, unknownResidue }; +} + +/** Table names in a SQLite file, excluding sqlite_* internals. */ +function readTables(dbPath) { + if (!fs.existsSync(dbPath)) return null; + const db = new DatabaseSync(dbPath, { readOnly: true }); + try { + const rows = db + .prepare("SELECT name FROM sqlite_master WHERE type = 'table' AND name NOT LIKE 'sqlite_%'") + .all(); + return new Set(rows.map((r) => r.name)); + } finally { + db.close(); + } +} + +function findDb(dataDir) { + const candidates = ["storage.sqlite", "omniroute.sqlite", "data.sqlite"]; + for (const name of candidates) { + const p = path.join(dataDir, name); + if (fs.existsSync(p)) return p; + } + const found = fs.readdirSync(dataDir).find((f) => f.endsWith(".sqlite")); + return found ? path.join(dataDir, found) : null; +} + +/** Boot an installed CLI and poll health. Returns { ok, version, failures, tail }. */ +async function bootAndProbe({ prefix, dataDir, port, expectVersion, label }) { + const binPath = path.join(prefix, "bin", "omniroute"); + if (!fs.existsSync(binPath)) { + return { ok: false, failures: [`${label}: bin not found at ${binPath}`], tail: [] }; + } + const child = spawn(binPath, ["serve", "--port", String(port)], { + env: { + ...process.env, + PORT: String(port), + DATA_DIR: dataDir, + JWT_SECRET: "install-upgrade-gate-secret-with-sufficient-length", + API_KEY_SECRET: "install-upgrade-gate-api-key-secret-long", + DISABLE_SQLITE_AUTO_BACKUP: "true", + OMNIROUTE_SKIP_SYSTEM_TRUST: "1", + }, + stdio: ["ignore", "pipe", "pipe"], + detached: true, + }); + + const tail = []; + const keepTail = (chunk) => { + tail.push(String(chunk)); + while (tail.length > 80) tail.shift(); + }; + child.stdout.on("data", keepTail); + child.stderr.on("data", keepTail); + let childExit = null; + child.on("exit", (code) => { + childExit = code ?? -1; + }); + + const deadline = Date.now() + BOOT_DEADLINE_MS; + let result = { ok: false, failures: [`${label}: never became healthy`], tail }; + while (Date.now() < deadline) { + if (childExit !== null) { + result = { ok: false, failures: [`${label}: exited with code ${childExit} before serving`], tail }; + break; + } + try { + const res = await fetch(`http://127.0.0.1:${port}/api/monitoring/health`); + const body = await res.json().catch(() => null); + if (res.status === 200 && body && typeof body === "object") { + const failures = []; + // `status` may legitimately report degraded (no providers configured) — the gate + // targets boot crashes and version mismatches, not health of a bare install. + if (expectVersion && body.version !== expectVersion) { + failures.push(`${label}: health reports version ${body.version}, expected ${expectVersion}`); + } + result = { ok: failures.length === 0, version: body.version, failures, tail }; + break; + } + } catch { + // not listening yet + } + await new Promise((r) => setTimeout(r, POLL_INTERVAL_MS)); + } + + try { + if (childExit === null) process.kill(-child.pid, "SIGTERM"); + } catch { + /* already gone */ + } + // Give the process a moment to flush and release the SQLite handle before we read the file. + await new Promise((r) => setTimeout(r, 3_000)); + return result; +} + +function npmInstallInto(prefix, spec) { + execFileSync("npm", ["install", "-g", "--prefix", prefix, "--no-audit", "--no-fund", spec], { + encoding: "utf8", + maxBuffer: 128 * 1024 * 1024, + }); +} + +function resolvePreviousVersion(current, explicit) { + if (explicit) return explicit; + const out = execFileSync("npm", ["view", "omniroute", "dist-tags.latest"], { encoding: "utf8" }); + const latest = out.trim(); + if (!latest) throw new Error("could not resolve omniroute@latest from npm"); + if (latest === current) { + // The version under test is already published (re-run of a shipped release): step back + // to the highest published version strictly below it. + const all = JSON.parse(execFileSync("npm", ["view", "omniroute", "versions", "--json"], { encoding: "utf8" })); + const stable = all.filter((v) => !/-(rc|alpha|beta|pre|next)/.test(v) && v !== current); + return stable[stable.length - 1]; + } + return latest; +} + +async function main() { + const ROOT = process.cwd(); + const args = process.argv.slice(2); + const fromIdx = args.indexOf("--from"); + const explicitFrom = fromIdx >= 0 ? args[fromIdx + 1] : null; + const skipUpgrade = args.includes("--skip-upgrade"); + + if (!fs.existsSync(path.join(ROOT, "dist", "server.js"))) { + console.error("[install-upgrade] dist/server.js missing — run `npm run build:cli` first"); + process.exit(2); + } + const version = JSON.parse(fs.readFileSync(path.join(ROOT, "package.json"), "utf8")).version; + const allowlist = loadAllowlist(ROOT); + const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-install-upgrade-")); + const failures = []; + const warnings = []; + + try { + log(`packing v${version}…`); + const packOut = execFileSync("npm", ["pack", "--json", "--pack-destination", tmp], { + cwd: ROOT, + encoding: "utf8", + maxBuffer: 128 * 1024 * 1024, + }); + const tarball = path.join(tmp, pickTarball(packOut)); + + // ---- Phase A: clean install ------------------------------------------------- + log("PHASE A — clean install of the packed tarball"); + const aPrefix = path.join(tmp, "a-prefix"); + const aData = path.join(tmp, "a-data"); + fs.mkdirSync(aData, { recursive: true }); + npmInstallInto(aPrefix, tarball); + const a = await bootAndProbe({ + prefix: aPrefix, + dataDir: aData, + port: pickPort(0), + expectVersion: version, + label: "clean", + }); + failures.push(...a.failures); + if (a.ok) log(`clean install healthy on v${a.version}`); + const aDb = findDb(aData); + const freshTables = aDb ? readTables(aDb) : null; + if (!freshTables) failures.push("clean: no SQLite database was created"); + else log(`clean install schema: ${freshTables.size} tables`); + + // ---- Phase B: upgrade over the previous published version ------------------- + let upgradedTables = null; + if (skipUpgrade) { + warn("PHASE B skipped (--skip-upgrade)"); + } else { + const previous = resolvePreviousVersion(version, explicitFrom); + log(`PHASE B — upgrade path: omniroute@${previous} → v${version}`); + const bPrefix = path.join(tmp, "b-prefix"); + const bData = path.join(tmp, "b-data"); + fs.mkdirSync(bData, { recursive: true }); + + npmInstallInto(bPrefix, `omniroute@${previous}`); + const before = await bootAndProbe({ + prefix: bPrefix, + dataDir: bData, + port: pickPort(1), + expectVersion: previous, + label: `previous(${previous})`, + }); + if (!before.ok) { + // A broken PREVIOUS version is not this release's fault — degrade to a warning so a + // historically bad publish cannot block the current one. + warnings.push(`previous version ${previous} did not boot cleanly — upgrade path unverified`); + for (const f of before.failures) warn(f); + } else { + const beforeDb = findDb(bData); + const beforeTables = beforeDb ? readTables(beforeDb) : new Set(); + log(`previous(${previous}) schema: ${beforeTables.size} tables — upgrading in place`); + + npmInstallInto(bPrefix, tarball); + const after = await bootAndProbe({ + prefix: bPrefix, + dataDir: bData, + port: pickPort(2), + expectVersion: version, + label: "upgraded", + }); + failures.push(...after.failures); + if (after.ok) log(`upgrade healthy on v${after.version}`); + + const afterDb = findDb(bData); + upgradedTables = afterDb ? readTables(afterDb) : null; + if (!upgradedTables) { + failures.push("upgraded: database disappeared after the upgrade"); + } else { + log(`upgraded schema: ${upgradedTables.size} tables`); + const dropped = [...beforeTables].filter((t) => !upgradedTables.has(t)); + if (dropped.length) { + failures.push(`upgrade DROPPED tables that existed before: ${dropped.join(", ")}`); + } + } + } + } + + // ---- Schema convergence ----------------------------------------------------- + if (freshTables && upgradedTables) { + const verdict = evaluateConvergence({ + freshTables, + upgradedTables, + residualAllowlist: allowlist.residualTables ?? {}, + }); + if (verdict.onlyUpgraded.length) { + warn(`residual tables present only after upgrade: ${verdict.onlyUpgraded.join(", ")}`); + } + failures.push(...verdict.failures); + if (verdict.ok) log("schema convergence OK (no new divergence)"); + } + + if (warnings.length) for (const w of warnings) warn(w); + if (failures.length) { + console.error(`[install-upgrade] FAIL — ${failures.length} problem(s):`); + for (const f of failures) console.error(` ✗ ${f}`); + process.exit(1); + } + log("PASS — clean install and upgrade path both boot; schema converges."); + process.exit(0); + } finally { + fs.rmSync(tmp, { recursive: true, force: true }); + } +} + +// Only run the (expensive) gate when invoked directly — importing this module for the pure +// helper above must not pack, install or boot anything. +if (process.argv[1] && path.resolve(process.argv[1]) === path.resolve(new URL(import.meta.url).pathname)) { + main().catch((err) => { + console.error(`[install-upgrade] crashed: ${err?.message ?? err}`); + process.exit(1); + }); +} diff --git a/tests/unit/check-install-upgrade-convergence.test.ts b/tests/unit/check-install-upgrade-convergence.test.ts new file mode 100644 index 0000000000..de3b74716a --- /dev/null +++ b/tests/unit/check-install-upgrade-convergence.test.ts @@ -0,0 +1,95 @@ +import assert from "node:assert/strict"; +import test from "node:test"; + +// @ts-expect-error — plain .mjs gate script, no type declarations by design +import { evaluateConvergence } from "../../scripts/check/check-install-upgrade.mjs"; + +/** + * The whole point of this gate is that the two directions of schema divergence are NOT + * equivalent, and the cheap symmetric check ("do the table sets match?") would either + * block every release on harmless residue or let a real upgrade bug through. + * + * Measured on 2026-07-30: a real 3.8.48 install on VPS .16 upgraded to 3.8.49 ended with + * 117 tables while a clean 3.8.49 install had 116 — the extra one being `cache_metrics`. + */ + +test("converged schemas pass", () => { + const v = evaluateConvergence({ + freshTables: ["a", "b", "c"], + upgradedTables: ["c", "b", "a"], + }); + assert.equal(v.ok, true); + assert.deepEqual(v.failures, []); + assert.deepEqual(v.onlyFresh, []); + assert.deepEqual(v.onlyUpgraded, []); +}); + +test("a table only a CLEAN install creates is ALWAYS a failure — upgraders lack structure", () => { + const v = evaluateConvergence({ + freshTables: ["a", "b", "new_feature_table"], + upgradedTables: ["a", "b"], + }); + assert.equal(v.ok, false); + assert.deepEqual(v.onlyFresh, ["new_feature_table"]); + assert.match(v.failures[0], /CLEAN install creates but an UPGRADE does not/); + assert.match(v.failures[0], /new_feature_table/); +}); + +test("that direction cannot be silenced by the residual allowlist", () => { + const v = evaluateConvergence({ + freshTables: ["a", "missing_on_upgrade"], + upgradedTables: ["a"], + // Even if someone lists it here, the dangerous direction must still fail. + residualAllowlist: { missing_on_upgrade: "please ignore me" }, + }); + assert.equal(v.ok, false); + assert.deepEqual(v.onlyFresh, ["missing_on_upgrade"]); +}); + +test("known residue (cache_metrics, the real 3.8.48→3.8.49 finding) passes but is reported", () => { + const v = evaluateConvergence({ + freshTables: ["a", "b"], + upgradedTables: ["a", "b", "cache_metrics"], + residualAllowlist: { cache_metrics: "measured 2026-07-30 on VPS .16" }, + }); + assert.equal(v.ok, true, "allowlisted residue must not block a release"); + assert.deepEqual(v.onlyUpgraded, ["cache_metrics"], "still surfaced so it stays visible"); + assert.deepEqual(v.unknownResidue, []); +}); + +test("UNKNOWN residue fails — a new divergence must not hide behind the allowlist", () => { + const v = evaluateConvergence({ + freshTables: ["a"], + upgradedTables: ["a", "cache_metrics", "surprise_table"], + residualAllowlist: { cache_metrics: "known" }, + }); + assert.equal(v.ok, false); + assert.deepEqual(v.unknownResidue, ["surprise_table"]); + assert.match(v.failures[0], /surprise_table/); + assert.doesNotMatch(v.failures[0], /cache_metrics/, "the known one must not be re-reported as new"); +}); + +test("both directions at once report both failures", () => { + const v = evaluateConvergence({ + freshTables: ["shared", "only_fresh"], + upgradedTables: ["shared", "only_upgraded"], + }); + assert.equal(v.ok, false); + assert.equal(v.failures.length, 2); + assert.deepEqual(v.onlyFresh, ["only_fresh"]); + assert.deepEqual(v.unknownResidue, ["only_upgraded"]); +}); + +test("accepts Sets as well as arrays (the gate passes Sets from sqlite_master)", () => { + const v = evaluateConvergence({ + freshTables: new Set(["a", "b"]), + upgradedTables: new Set(["a", "b"]), + }); + assert.equal(v.ok, true); +}); + +test("empty/missing inputs do not crash", () => { + const v = evaluateConvergence({ freshTables: undefined, upgradedTables: undefined }); + assert.equal(v.ok, true); + assert.deepEqual(v.onlyFresh, []); +}); diff --git a/tests/unit/npm-publish-artifact-provenance.test.ts b/tests/unit/npm-publish-artifact-provenance.test.ts new file mode 100644 index 0000000000..392e469a5b --- /dev/null +++ b/tests/unit/npm-publish-artifact-provenance.test.ts @@ -0,0 +1,90 @@ +/** + * Supply-chain guard for the `next-build` fast path in npm-publish.yml. + * + * The publish job restores a CI-built standalone tree and ships it as the npm tarball. + * The run it restores from is picked by querying the runs API for `head_sha`. That query + * also returns `pull_request` runs from FORKS — they execute in this repository's context + * and upload their own `next-build`, built from fork-controlled source. Measured on + * 2026-07-30: 57 runs in this repo have a `head_repository` other than the repo itself. + * + * Selecting on name + conclusion alone therefore trusts bytes by coincidence of commit + * SHA. The `head_repository.full_name == env.REPO` clause is what makes the provenance + * explicit. Raised as CodeQL `actions/artifact-poisoning/critical`. + * + * This is a guard, not a reproduction: nothing here can exercise a real poisoning attempt. + * It asserts the clause cannot be dropped in a future edit without a test turning red. + */ +import test from "node:test"; +import assert from "node:assert/strict"; +import fs from "node:fs"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; + +const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "../.."); + +function readPublishWorkflow(): string { + return fs.readFileSync(path.join(repoRoot, ".github/workflows/npm-publish.yml"), "utf-8"); +} + +/** The `run:` body of the step whose `name:` matches, at any indentation. */ +function extractStep(yaml: string, stepName: string): string { + const lines = yaml.split("\n"); + const startIdx = lines.findIndex((l) => l.includes(`- name: ${stepName}`)); + assert.ok(startIdx !== -1, `npm-publish.yml must define the "${stepName}" step`); + const indent = lines[startIdx].indexOf("- name:"); + const block: string[] = []; + for (let i = startIdx + 1; i < lines.length; i++) { + // The next list item at the same indentation ends this step. + if (lines[i].slice(indent).startsWith("- ")) break; + block.push(lines[i]); + } + return block.join("\n"); +} + +const ARTIFACT_STEP = "Reuse CI's next-build artifact (skips the heavy rebuild)"; + +test("the artifact fast path only trusts runs from THIS repository", () => { + const step = extractStep(readPublishWorkflow(), ARTIFACT_STEP); + + assert.match( + step, + /head_repository\.full_name == env\.REPO/, + "the run selection must require the run to originate from this repository — without it, " + + "a fork's pull_request run supplies the bytes that get published to npm" + ); + // Provenance is necessary but not sufficient: tree-equality still comes from head_sha. + assert.match( + step, + /head_sha=\$HEAD_SHA/, + "the run must still be matched on head_sha — that is the tree-equality guarantee" + ); + assert.match(step, /\.conclusion == "success"/, "only a successful run may be restored"); +}); + +test("REPO is passed through env, never interpolated into the script body", () => { + const step = extractStep(readPublishWorkflow(), ARTIFACT_STEP); + + assert.match(step, /REPO:\s*\$\{\{\s*github\.repository\s*\}\}/, "REPO must come from env:"); + // Hard rule #13 / zizmor template-injection: no ${{ }} inside the run: body. + const runBody = step.slice(step.indexOf("run: |")); + assert.doesNotMatch( + runBody, + /\$\{\{/, + "the run: body must not interpolate any ${{ }} expression — pass values via env:" + ); +}); + +test("a miss falls through to a real build instead of publishing an empty tree", () => { + const step = extractStep(readPublishWorkflow(), ARTIFACT_STEP); + + assert.match( + step, + /continue-on-error:\s*true/, + "the fast path is best-effort — a miss must not fail the publish" + ); + assert.match( + step, + /falling back to a full build/, + "a miss must say so, so a silent no-op cannot look like a restore" + ); +});