Files
OmniRoute/tests/unit/check-install-upgrade-convergence.test.ts
Diego Rodrigues de Sa e Souza 7eca04fd12 feat(ci): gate the publish on clean-install AND upgrade-over-previous (#8953)
* feat(ci): gate the publish on clean-install AND upgrade-over-previous

`check:pack-boot` proves a fresh install boots. It does not prove the path that actually
broke us: installing over an existing version, where ~110 SQLite migrations run against a
populated database. v3.8.48 shipped as a hotfix because the published 3.8.47 crashed on
boot, and the v3.8.49 upgrade path was only ever exercised end-to-end by hand — on VPS .16,
against a real 3.8.48 install with a 165 MB database, AFTER publishing. That is backwards.

New gate (`scripts/check/check-install-upgrade.mjs`), wired into npm-publish.yml as step 12,
BEFORE `npm stage publish` — so a broken upgrade never reaches the registry and a staged
package that is never approved simply expires, with no `npm deprecate` needed:

- Phase A: fresh prefix + fresh DATA_DIR, install the packed tarball, boot, health.
- Phase B: fresh prefix + fresh DATA_DIR, install the PREVIOUS published version, boot it
  (creates + migrates the DB), stop, install the tarball over the SAME prefix, boot against
  the SAME DATA_DIR. Asserts no table present before the upgrade was dropped.
- Schema convergence, and its DIRECTION is the whole point:
    fresh − upgraded ≠ ∅  → FAIL. Structure a clean install creates but an upgrade does not
                            means every existing user is missing it. Not allowlistable.
    upgraded − fresh ≠ ∅  → residue; fails only when NEW (allowlist carries the known ones).

A naive symmetric check would either block every release on harmless residue or, if relaxed,
let the dangerous direction through. Measured on VPS .16 (2026-07-30): a real 3.8.48 install
upgraded to 3.8.49 ended with 117 tables against 116 for a clean 3.8.49 install — the extra
being `cache_metrics`, recorded in config/quality/install-upgrade-allowlist.json with the
measurement. Both installs healthy, zero `no such table` in 150 log lines.

`evaluateConvergence` is exported and pure so the asymmetry is testable without packing,
installing or booting anything (same reason check-test-masking exports its helpers):
tests/unit/check-install-upgrade-convergence.test.ts, 8 cases, ~6ms.

A previous version that fails to boot degrades to a warning — a historically bad publish
must not block the current one. Uses node:sqlite (Node 24, already the publish job's
runtime): no new dependency.

* fix(ci): require the reused next-build artifact to come from this repository

CodeQL raised actions/artifact-poisoning/critical on the `next-build` fast path
this PR builds on (#8941). The finding is real and it sits on the path that
produces the published npm tarball.

The step picks a CI run by querying the runs API for `head_sha` and filtering on
`name == "CI" and conclusion == "success"`. That query also returns
`pull_request` runs from FORKS: they execute in this repository's context and
upload their own `next-build`, built from fork-controlled source. Measured
today, 57 runs in this repo have a `head_repository` other than the repo itself.
So the selection trusted bytes by coincidence of commit SHA — anything that made
a fork's head commit coincide with the publish commit could put attacker-built
bytes on npm.

Adds `and .head_repository.full_name == env.REPO` to the selection. Provenance
is now explicit; `head_sha` still carries tree-equality. Verified against the
live API using the expression extracted from the workflow itself — the same
single run (30518663668) is selected either way for the current tip, so the fast
path keeps working while every fork run is excluded.

Not a dismissal (hard rule #14) — the clause removes the flagged trust.

    node --import tsx/esm --test tests/unit/npm-publish-artifact-provenance.test.ts
    # 3 pass, 0 fail        (base: 2 pass, 1 fail)

---------

Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
2026-07-30 09:07:57 -03:00

96 lines
3.6 KiB
TypeScript

import assert from "node:assert/strict";
import test from "node:test";
// @ts-expect-error — plain .mjs gate script, no type declarations by design
import { evaluateConvergence } from "../../scripts/check/check-install-upgrade.mjs";
/**
* The whole point of this gate is that the two directions of schema divergence are NOT
* equivalent, and the cheap symmetric check ("do the table sets match?") would either
* block every release on harmless residue or let a real upgrade bug through.
*
* Measured on 2026-07-30: a real 3.8.48 install on VPS .16 upgraded to 3.8.49 ended with
* 117 tables while a clean 3.8.49 install had 116 — the extra one being `cache_metrics`.
*/
test("converged schemas pass", () => {
const v = evaluateConvergence({
freshTables: ["a", "b", "c"],
upgradedTables: ["c", "b", "a"],
});
assert.equal(v.ok, true);
assert.deepEqual(v.failures, []);
assert.deepEqual(v.onlyFresh, []);
assert.deepEqual(v.onlyUpgraded, []);
});
test("a table only a CLEAN install creates is ALWAYS a failure — upgraders lack structure", () => {
const v = evaluateConvergence({
freshTables: ["a", "b", "new_feature_table"],
upgradedTables: ["a", "b"],
});
assert.equal(v.ok, false);
assert.deepEqual(v.onlyFresh, ["new_feature_table"]);
assert.match(v.failures[0], /CLEAN install creates but an UPGRADE does not/);
assert.match(v.failures[0], /new_feature_table/);
});
test("that direction cannot be silenced by the residual allowlist", () => {
const v = evaluateConvergence({
freshTables: ["a", "missing_on_upgrade"],
upgradedTables: ["a"],
// Even if someone lists it here, the dangerous direction must still fail.
residualAllowlist: { missing_on_upgrade: "please ignore me" },
});
assert.equal(v.ok, false);
assert.deepEqual(v.onlyFresh, ["missing_on_upgrade"]);
});
test("known residue (cache_metrics, the real 3.8.48→3.8.49 finding) passes but is reported", () => {
const v = evaluateConvergence({
freshTables: ["a", "b"],
upgradedTables: ["a", "b", "cache_metrics"],
residualAllowlist: { cache_metrics: "measured 2026-07-30 on VPS .16" },
});
assert.equal(v.ok, true, "allowlisted residue must not block a release");
assert.deepEqual(v.onlyUpgraded, ["cache_metrics"], "still surfaced so it stays visible");
assert.deepEqual(v.unknownResidue, []);
});
test("UNKNOWN residue fails — a new divergence must not hide behind the allowlist", () => {
const v = evaluateConvergence({
freshTables: ["a"],
upgradedTables: ["a", "cache_metrics", "surprise_table"],
residualAllowlist: { cache_metrics: "known" },
});
assert.equal(v.ok, false);
assert.deepEqual(v.unknownResidue, ["surprise_table"]);
assert.match(v.failures[0], /surprise_table/);
assert.doesNotMatch(v.failures[0], /cache_metrics/, "the known one must not be re-reported as new");
});
test("both directions at once report both failures", () => {
const v = evaluateConvergence({
freshTables: ["shared", "only_fresh"],
upgradedTables: ["shared", "only_upgraded"],
});
assert.equal(v.ok, false);
assert.equal(v.failures.length, 2);
assert.deepEqual(v.onlyFresh, ["only_fresh"]);
assert.deepEqual(v.unknownResidue, ["only_upgraded"]);
});
test("accepts Sets as well as arrays (the gate passes Sets from sqlite_master)", () => {
const v = evaluateConvergence({
freshTables: new Set(["a", "b"]),
upgradedTables: new Set(["a", "b"]),
});
assert.equal(v.ok, true);
});
test("empty/missing inputs do not crash", () => {
const v = evaluateConvergence({ freshTables: undefined, upgradedTables: undefined });
assert.equal(v.ok, true);
assert.deepEqual(v.onlyFresh, []);
});