Files
OmniRoute/tests/unit/ops-scripts.test.ts
Diego Rodrigues de Sa e Souza 5ac4637816 docs: reorganize docs/ + accuracy audit + drop Node 20 (rebased onto current release tip) (#5262)
* docs: move superpowers/research artifacts to isolated _tasks repo + docs tree cleanup

- Move docs/superpowers/{plans,specs} and docs/research/* into the gitignored,
  separately-versioned _tasks/ repo; untrack the two tracked research design docs.
- Add CLAUDE.md "Planning & Research Artifacts" section overriding the superpowers
  default save paths (docs/... -> _tasks/...); align REPOSITORY_MAP and
  DOCUMENTATION_OVERHAUL_PLAN with the new convention.
- Drop 4 now-obsolete /api/discovery/* entries from check-docs-symbols allowlist
  (stale-enforcement) and refresh code/spec path comments to _tasks/...
- Sweeps in concurrent docs-tree restructuring (root-level provider/guide docs,
  compression spec cleanup, .mcp.json.example removal).

* docs: reorganize docs/ tree + fix stale facts across ~26 docs

Phase A — reorganization:
- Move 7 orphan root docs into subfolders (providers/ created; TIERS+USAGE_QUOTA→guides/;
  plugins+PLUGIN_SDK→frameworks/); delete 8 obsolete/redundant docs (SUBMIT_PR superseded
  by CONTRIBUTING; DOCUMENTATION_OVERHAUL_PLAN; INCIDENT_RESPONSE/PERF_BUDGETS/THREAT_MODEL;
  3 ops snapshots). Rebuild README index (was missing ~40 files) + per-folder meta.json nav.
- Clean 14 dangling doc-path references in bin/ ops scripts, scripts/, workflow, tests;
  fix the dockerignore-docs-coverage required-docs path (PROVIDERS→providers/CLAUDE_WEB).

Phase B — content accuracy (verified against code, not the audit summary):
- Functional: ENVIRONMENT flag defaults (INPUT_SANITIZER/MCP_ENFORCE_SCOPES=true,
  COMPRESS_DESCRIPTIONS=false, dynamic heap); MCP-SERVER notion tool names (omniroute_*→
  notion_*) + counts 87→94; coverage gate 75/70→60/60/60/60 (RELEASE_CHECKLIST, COVERAGE_PLAN,
  ERROR_SANITIZATION, CONTRIBUTING); pre-push hook description; regenerate PROVIDER_REFERENCE (237).
- Count drift: providers 237, executors 70, migrations 106, db modules 94, oauth 19,
  strategies 17, MCP 94, flags 38, TS 6.0, open-sse ~900/services 294 across architecture/
  frameworks/ops docs; AUTO-COMBO 9→12 factors w/ correct DEFAULT_WEIGHTS; REASONING +2
  patterns; STEALTH UA defaults; AGENT_PROTOCOLS +cursor-cloud/list-capabilities;
  LANGUAGE_PACKS +id pack.
- Kept Node 20 (runtime guard accepts 20.20.2+; only engines is stricter) and MCP scopes=13
  (mcpScopes.ts) — both were correct in the docs; corrected only the attribution.

* docs: finish content refresh — compression engines, CLI_TOKEN merge, metadata sweep

- Compression: document the additional built-in engines (CCR, headroom, ionizer,
  session-dedup) in COMPRESSION_ENGINES; clarify LLMLingua-2 is the ultra-mode SLM
  backend + cross-ref the extra engines in EXTENDING_COMPRESSION; add the id
  (Indonesian) language pack to LANGUAGE_PACKS.
- AUTO-COMBO: replace the orphan 'How tiers fit' weight table (stale weights) with a
  pointer to the canonical 12-factor DEFAULT_WEIGHTS table.
- Security: merge CLI_TOKEN_AUTH.md (legacy 32-char SHA-256 format) into CLI_TOKEN.md
  as a 'Legacy format — still accepted' section (server accepts both HMAC + legacy),
  delete CLI_TOKEN_AUTH.md, drop it from the index + security nav.
- Metadata: bump stale frontmatter (version/lastUpdated) to 3.8.40/2026-06-28 across the
  doc set audited this pass, and normalize the in-body 'Last updated' header lines to match.

* fix(runtime): drop Node 20 from supported range + align all docs/diagrams/counts

- Node minimum is now 22 (aligned with package.json engines). SUPPORTED_NODE_RANGE in
  src/shared/utils/nodeRuntimeSupport.ts (and the bin/ mirror) drops the 20.x line →
  '>=22.22.2 <23 || >=24.0.0 <27'; getNodeRuntimeSupport now rejects Node 20 as
  unsupported-major. Test updated (TDD): node-runtime-support.test.ts asserts Node 20
  rejected. Docs aligned (TROUBLESHOOTING ×2, TERMUX, RELEASE_CHECKLIST, CODEBASE,
  CLI-TOOLS, README, llm.txt + 42 i18n llm.txt mirrors, skills/cli-serve).
- Diagrams regenerated: mcp-tools-87 -> mcp-tools-94 (34 base + pool 6 = 94) and
  auto-combo-9factor -> auto-combo-12factor (correct DEFAULT_WEIGHTS); SVGs re-rendered
  via mermaid-cli; doc refs + diagrams/README updated; fixed a pre-existing broken
  resilience-3layers image path.
- CLAUDE.md + AGENTS.md aligned to real counts (237 providers, 94 MCP tools / 34 base,
  106 migrations, 94 db modules, 12-factor auto-combo, 17 strategies); README provider
  count 231 -> 237; executor count corrected to 68 (provider executors, excl base/index)
  and OAuth to 18 across architecture docs. check:docs-all now passes (0 strict drift,
  0 broken links); removed dead .mcp.json.example doc link.

* fix(services): update installer Node hint to >=22.22.2 (aligned with dropped Node 20)

* docs: realign counts to current release tip after rebase

The release tip advanced while this work was in flight (Gemini CLI provider/executor
removed by #5246, plus other PRs). Re-counted against the current code and updated:
providers 237->236, executors 68->67, OAuth modules 18->17, open-sse services 294->298;
regenerated PROVIDER_REFERENCE.md (236). check:docs-all passes (0 strict drift).

* docs(changelog) + i18n: record Node 20 drop + fix nodeIncompatibleHint

- CHANGELOG: add [3.8.40] entries for the Node 20.x removal (runtime) and the docs
  reorganization/accuracy audit.
- i18n: nodeIncompatibleHint across all 42 locales no longer lists Node 20.x as
  supported (ASCII + CJK full-width variants), aligned with the dropped Node 20.

* fix(docs): repair CI breakages from the doc moves

- test: cli-plugin-system asserted docs/dev/plugins.md exists; the file moved to
  docs/frameworks/PLUGINS.md — point the test at the new path (Unit fast-path 2/2 fix).
- frontmatter: PLUGINS.md and PLUGIN_SDK.md moved into the fumadocs-indexed
  docs/frameworks/ which requires a 'title' frontmatter; the missing frontmatter
  failed the Next.js MDX build (dast-smoke 'invalid frontmatter'). Added frontmatter
  to both, plus the providers/ docs (consistency; that folder is not indexed).
2026-06-28 22:07:18 -03:00

185 lines
7.0 KiB
TypeScript

/**
* tests/unit/ops-scripts.test.ts
*
* Contract tests for the self-hoster incident-recovery / cold-start ops scripts
* under bin/:
* rollback.sh · snapshot-data.sh · restore-data.sh · restore-policies.sh ·
* cold-start-bench.sh
*
* These scripts touch deploys and the SQLite store, so the suite pins down the
* safety contract rather than full ops behavior: every script is executable
* bash with strict mode, prints usage on --help, the restore commands refuse to
* run without a snapshot id, and a snapshot→restore round-trip works while the
* non-interactive TTY guard blocks an unattended destructive restore.
*/
import { describe, it } from "node:test";
import assert from "node:assert/strict";
import { spawnSync } from "node:child_process";
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
const ROOT = path.resolve(import.meta.dirname, "..", "..");
const BIN = path.join(ROOT, "bin");
const SCRIPTS = [
"rollback.sh",
"snapshot-data.sh",
"restore-data.sh",
"restore-policies.sh",
"cold-start-bench.sh",
];
const hasSqlite3 = spawnSync("sqlite3", ["--version"], { stdio: "ignore" }).status === 0;
/** Run a bin/ script with a NON-tty stdin (so the TTY guard engages). */
function runScript(script: string, args: string[], env: Record<string, string> = {}) {
return spawnSync("bash", [path.join(BIN, script), ...args], {
encoding: "utf8",
stdio: ["ignore", "pipe", "pipe"],
env: { ...process.env, ...env },
});
}
describe("ops runbook scripts (bin/*.sh)", () => {
it("every script exists, is executable, and uses bash + strict mode", () => {
for (const s of SCRIPTS) {
const p = path.join(BIN, s);
assert.ok(fs.existsSync(p), `${s} is missing`);
assert.ok(fs.statSync(p).mode & 0o111, `${s} is not executable (chmod +x)`);
const body = fs.readFileSync(p, "utf8");
assert.ok(body.startsWith("#!/usr/bin/env bash"), `${s} missing bash shebang`);
assert.ok(body.includes("set -euo pipefail"), `${s} missing 'set -euo pipefail'`);
}
});
it("the shared helper and every script pass `bash -n` (syntax check)", () => {
for (const s of [...SCRIPTS, "_ops-common.sh"]) {
const r = spawnSync("bash", ["-n", path.join(BIN, s)], { encoding: "utf8" });
assert.equal(r.status, 0, `${s} has a syntax error: ${r.stderr}`);
}
});
it("--help exits 0 with a usage banner for every script", () => {
for (const s of SCRIPTS) {
const r = runScript(s, ["--help"]);
assert.equal(r.status, 0, `${s} --help exited ${r.status}: ${r.stderr}`);
assert.match(r.stdout, /Usage:/, `${s} --help printed no usage banner`);
}
});
it("restore scripts refuse to run without a snapshot id", () => {
for (const s of ["restore-data.sh", "restore-policies.sh"]) {
const r = runScript(s, []);
assert.notEqual(r.status, 0, `${s} should fail without an id`);
assert.match(r.stderr, /snapshot id required/, `${s} wrong error: ${r.stderr}`);
}
});
it("snapshot → restore-data round-trips, and the TTY guard blocks unattended restores", async () => {
const dataDir = fs.mkdtempSync(path.join(os.tmpdir(), "omni-ops-"));
try {
const Database = (await import("better-sqlite3")).default;
const dbPath = path.join(dataDir, "storage.sqlite");
let db = new Database(dbPath);
db.exec(
"CREATE TABLE api_keys (id TEXT PRIMARY KEY, name TEXT);" +
"INSERT INTO api_keys VALUES ('k1','orig');"
);
db.close();
const env = { DATA_DIR: dataDir };
const snap = runScript("snapshot-data.sh", ["--label", "test"], env);
assert.equal(snap.status, 0, `snapshot failed: ${snap.stderr}`);
const id = snap.stdout.trim();
assert.ok(id, "snapshot id not printed on stdout");
assert.ok(
fs.existsSync(path.join(dataDir, "db_backups", `snapshot_${id}`, "storage.sqlite")),
"snapshot dir not created"
);
// Mutate the live DB so a successful restore is observable.
db = new Database(dbPath);
db.exec("UPDATE api_keys SET name='changed' WHERE id='k1';");
db.close();
// Guard: non-interactive restore WITHOUT --yes must refuse (nothing destroyed).
const blocked = runScript("restore-data.sh", [id], env);
assert.notEqual(blocked.status, 0, "restore without --yes should be blocked");
assert.match(blocked.stderr, /TTY/, `expected TTY guard, got: ${blocked.stderr}`);
db = new Database(dbPath);
assert.equal(
(db.prepare("SELECT name FROM api_keys WHERE id='k1'").get() as { name: string }).name,
"changed",
"blocked restore must not have changed the DB"
);
db.close();
// With --yes it reverts to the snapshot.
const ok = runScript("restore-data.sh", [id, "--yes"], env);
assert.equal(ok.status, 0, `restore failed: ${ok.stderr}`);
db = new Database(dbPath);
assert.equal(
(db.prepare("SELECT name FROM api_keys WHERE id='k1'").get() as { name: string }).name,
"orig",
"restore did not revert the row"
);
db.close();
} finally {
fs.rmSync(dataDir, { recursive: true, force: true });
}
});
it(
"restore-policies replaces only api_key* tables, preserving other tables",
{ skip: hasSqlite3 ? false : "sqlite3 CLI not installed" },
async () => {
const dataDir = fs.mkdtempSync(path.join(os.tmpdir(), "omni-pol-"));
try {
const Database = (await import("better-sqlite3")).default;
const dbPath = path.join(dataDir, "storage.sqlite");
let db = new Database(dbPath);
db.exec(
"CREATE TABLE api_keys (id TEXT PRIMARY KEY, name TEXT);" +
"INSERT INTO api_keys VALUES ('k1','orig');" +
"CREATE TABLE sessions (id TEXT PRIMARY KEY);" +
"INSERT INTO sessions VALUES ('s-old');"
);
db.close();
const env = { DATA_DIR: dataDir };
const id = runScript("snapshot-data.sh", [], env).stdout.trim();
assert.ok(id, "snapshot id not printed");
// Change BOTH a policy table and a non-policy table after the snapshot.
db = new Database(dbPath);
db.exec(
"UPDATE api_keys SET name='changed' WHERE id='k1';" +
"INSERT INTO sessions VALUES ('s-new');"
);
db.close();
const r = runScript("restore-policies.sh", [id, "--yes"], env);
assert.equal(r.status, 0, `restore-policies failed: ${r.stderr}`);
db = new Database(dbPath);
// Policy table reverted…
assert.equal(
(db.prepare("SELECT name FROM api_keys WHERE id='k1'").get() as { name: string }).name,
"orig",
"api_keys policy was not restored"
);
// …non-policy table left intact (live usage not rewound).
assert.equal(
(db.prepare("SELECT COUNT(*) c FROM sessions").get() as { c: number }).c,
2,
"non-policy table must not be touched by restore-policies"
);
db.close();
} finally {
fs.rmSync(dataDir, { recursive: true, force: true });
}
}
);
});