pjandro/OmniRoute
1
0
Fork 0
mirror of https://github.com/diegosouzapw/OmniRoute.git synced 2026-09-13 18:32:12 +03:00
Code Issues Packages Projects Releases Wiki Activity

chore(ci): gate the two blind spots that let silent debt accumulate

Browse Source 
check:vitest-exclusions — a file in vitest.config.ts's exclude list is a test that
does not run, indistinguishable from one that does not exist except that it looks
like coverage. 62 files accumulated behind '// #8618 — pre-existing failure',
while #8618 itself was closed on 2026-08-11 and the list grew from 45 to 62. The
gate requires every exclusion to name a tracker and to appear in
config/quality/vitest-exclusions.json with its measured status, so growth is a
reviewable diff in a dedicated file. It does not re-run the tests — that is a
periodic job, and the inventory records when each was last measured.

check-new-key-coverage — sibling of check-ui-value-drift. That one catches a
rewritten English value leaving stale translations; this one catches a new English
key that some locales never received. check-ui-keys-coverage cannot: it is a
percentage floor per locale, and 11 absent keys out of ~13,000 leaves it at 99.9%.
Proven retroactively against the real incident — with BASE_REF set before the
Phase 3 merge it flags all 11 canvas keys across exactly the nine EU locales that
missed them, plus 3 keys from other features with the same problem.

Both are diff-aware against the merge base, so pre-existing gaps stay frozen and
neither needed a migration. Both wired into ci.yml and documented in
docs/architecture/QUALITY_GATES.md, with 14 tests covering the pure cores.

Refs #13204
This commit is contained in:
diegosouzapw
2026-09-10 18:25:24 -03:00
parent e3493c6de2
commit 4a4545b662
9 changed files with 640 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
.github/workflows/ci.yml vendored
View File

@@ -144,6 +144,12 @@ jobs:
- run: npm run check:test-discovery
- run: npm run check:radar-sentinels
- run: npm run check:tracked-artifacts
# A test parked in vitest.config.ts's exclude list does not run, and looks like
# coverage to whoever reads the tree. 62 files accumulated behind a comment pointing
# at #8618 — closed in August while the list grew to 62; 51 of them passed when
# finally measured (#13204). This gate requires every exclusion to name a tracker and
# to appear in config/quality/vitest-exclusions.json, so the debt stays reviewable.
- run: npm run check:vitest-exclusions
# (gap 30) Also lives in quality.yml's PR-only "Merge integrity" job — because the
# CHANGELOG half of that job needs a base to diff against. This half does NOT: the
# generator either reproduces the committed SKILL.md files or it does not.
@@ -515,6 +521,16 @@ jobs:
env:
BASE_REF: ${{ github.base_ref && format('origin/{0}', github.base_ref) || '' }}
run: node scripts/i18n/check-ui-value-drift.mjs
# Sibling of the drift gate above. That one catches an English value that was
# REWRITTEN; this one catches an English key that was ADDED while some locales never
# got it. The coverage gate at the top of this job cannot: it is a percentage per
# locale, and 11 absent keys out of ~13,000 leaves coverage at 99.9%. Incident: the
# Phase 3 canvas keys were translated across the 42 locales that existed, then the EU
# batch (#13044) took the repo to 51 and the nine newcomers shipped untranslated.
- name: i18n new-key coverage (a new key must reach every locale)
env:
BASE_REF: ${{ github.base_ref && format('origin/{0}', github.base_ref) || '' }}
run: node scripts/i18n/check-new-key-coverage.mjs
# #8038: cheap glossary/protected-terms consistency gate —
# complements i18n-ui-coverage (key parity) and the ICU `i18n` job below

8
changelog.d/maintenance/gates-for-silent-debt.md Normal file
View File

@@ -0,0 +1,8 @@
- **chore(ci):** two gates that close the blind spots behind the exclusions above.
`check:vitest-exclusions` requires every Vitest exclusion to name a tracking issue and to
appear in `config/quality/vitest-exclusions.json` — the previous list grew to 62 files behind
a comment pointing at an issue that had been closed for a month. `check-new-key-coverage`
requires a key newly added to `en.json` to reach every locale; the existing coverage gate is a
percentage floor per locale, so eleven absent keys out of ~13,000 left it at 99.9% while a
whole feature shipped untranslated in nine languages. Both are diff-aware, so pre-existing
debt stays frozen and neither needed a migration to turn on.

72
config/quality/vitest-exclusions.json Normal file
View File

@@ -0,0 +1,72 @@
{
"_comment": "Inventário dos arquivos de teste excluídos do Vitest. Toda entrada precisa de uma issue de rastreio ABERTA. Gate: npm run check:vitest-exclusions. Contexto: #13204.",
"_measured": "2026-09-10 — cada arquivo rodado isoladamente com as exclusões removidas",
"excluded": [
{
"file": "tests/unit/ui/request-logger-autorefresh-visibility-3972.test.tsx",
"issue": "#13204",
"measured": "2026-09-10",
"status": "Tests 1 failed | 5 passed (6)"
},
{
"file": "src/app/(dashboard)/dashboard/webhooks/__tests__/webhook-wizard.test.tsx",
"issue": "#13204",
"measured": "2026-09-10",
"status": "Tests 1 failed | 6 passed (7)"
},
{
"file": "tests/unit/ui/logs-page-detail-modal-reopen-on-close.test.tsx",
"issue": "#13204",
"measured": "2026-09-10",
"status": "Tests 2 failed (2)"
},
{
"file": "tests/unit/ui/agent-card.test.tsx",
"issue": "#13204",
"measured": "2026-09-10",
"status": "Tests 3 failed | 1 passed (4)"
},
{
"file": "src/app/(dashboard)/dashboard/endpoint/__tests__/ApiEndpointsTab.test.tsx",
"issue": "#13204",
"measured": "2026-09-10",
"status": "Tests 2 failed | 1 passed (3)"
},
{
"file": "src/app/(dashboard)/dashboard/cache/__tests__/CacheTrends.test.tsx",
"issue": "#13204",
"measured": "2026-09-10",
"status": "Tests 7 failed | 6 passed (13)"
},
{
"file": "src/app/(dashboard)/dashboard/cache/__tests__/IdempotencyLayer.test.tsx",
"issue": "#13204",
"measured": "2026-09-10",
"status": "Tests 9 failed | 4 passed (13)"
},
{
"file": "src/app/(dashboard)/dashboard/cache/__tests__/CachePerformance.test.tsx",
"issue": "#13204",
"measured": "2026-09-10",
"status": "Tests 10 failed | 4 passed (14)"
},
{
"file": "src/app/(dashboard)/dashboard/discovery/__tests__/DiscoveryPageClient.test.tsx",
"issue": "#13204",
"measured": "2026-09-10",
"status": "Tests 2 failed | 1 passed (3)"
},
{
"file": "tests/unit/ui/combos-page-smoke.test.tsx",
"issue": "#13204",
"measured": "2026-09-10",
"status": "Tests 1 failed (1)"
},
{
"file": "tests/unit/ui/evals-tab-smoke.test.tsx",
"issue": "#13204",
"measured": "2026-09-10",
"status": "Tests 1 failed (1)"
}
]
}

39
docs/architecture/QUALITY_GATES.md
View File

@@ -71,6 +71,7 @@ Runs on every PR to `main`. Blocks merge on failure.
| `check:lockfile` | `package-lock.json` integrity — https registry, integrity hashes, no host overrides | Yes |
| `check:licenses` | SPDX license allowlist for production dependencies | Yes |
| `check:tracked-artifacts` | No build artifacts / committed `node_modules` symlinks (also runs in husky pre-commit; pre-push is intentionally light — #6716) | Yes |
| `check:vitest-exclusions` | Every Vitest exclusion names a tracking issue and appears in `config/quality/vitest-exclusions.json` (#13204) | Yes |
| `check:file-size` | No source file exceeds the per-extension cap (ratchet: frozen large files in `frozen` list) | Yes |
| `check:error-helper` | Error responses in executors/handlers use `buildErrorBody()` / `sanitizeErrorMessage()` (Hard Rule #12) | Yes |
| `check:migration-numbering` | Migration SQL files are sequentially numbered, no gaps or duplicates | Yes |
@@ -143,6 +144,7 @@ Runs on every PR to `main`. Blocks merge on failure.
| --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------ |
| `check-ui-keys-coverage` (inline) | UI i18n key coverage is ≥ 65% | Yes |
| `check-ui-value-drift` (inline) | A rewritten English **value** leaves no stale translation behind | Yes |
| `check-new-key-coverage` (inline) | A **new** English key reaches every locale | Yes |
| `check-translation-ratio` | Real-translation ratio per locale (identical-to-English / placeholder / missing leaves outside the allowlist) must not exceed `config/quality/i18n-translation-baseline.json` + slack | **Advisory** |
Needs `fetch-depth: 0` — the value-drift gate diffs `en.json` against the merge base.
@@ -533,3 +535,40 @@ several "obvious" merges turned out to hide debt and are **not** clean drop-ins.
## Related Documentation
- Supply-chain (provenance, SBOM, Trivy, Scorecard): [`docs/security/SUPPLY_CHAIN.md`](../security/SUPPLY_CHAIN.md)
#### `check-new-key-coverage` — new-key i18n gate
Sibling of `check-ui-value-drift`. That one catches an English value that was **rewritten**
while its translations were left behind; this one catches an English key that was **added**
while some locales never received it.
`check-ui-keys-coverage` cannot see this class: it enforces a percentage floor per locale, and
eleven absent keys out of ~13,000 leaves coverage at 99.9%. A percentage per language cannot
express "this feature shipped untranslated" — an entire feature can land in a new locale with no
text and never move the number.
The incident it encodes: Phase 3 of the Orchestration Canvas translated its eleven keys across
the 42 locales that existed at the time. Hours later the EU-language batch (#13044) took the repo
to 51 locales, and the nine newcomers (`el`, `et`, `ga`, `hr`, `lt`, `lv`, `mt`, `sl`, `sr`) never
received them. `deepMergeFallback` substitutes English for an absent key, so the failure mode was
untranslated UI rather than blank UI — real, and silent by construction.
Like its sibling it is **diff-aware**, comparing English at the merge base against the working
tree, so pre-existing gaps stay frozen and the gate needed no migration to turn on. Escape hatch:
`__MISSING__:<english>` defers a translation while keeping the runtime correct. `vi` bans
placeholders (`tests/unit/i18n-vi-completeness.test.ts`) and needs a real translation.
#### `check-vitest-exclusions` — parked-test gate
A file in `vitest.config.ts`'s `exclude` list is a test that does not run, and it looks like
coverage to whoever reads the tree. Sixty-two files accumulated behind the comment
`// #8618 — pre-existing failure; remove this exclusion when fixed`. Issue #8618 was closed on
2026-08-11 while the list it tracked grew from 45 entries to 62, each new one inheriting a comment
pointing at a dead issue. When the list was finally measured file by file (#13204), **51 of the 62
passed against the current tree with no source change**.
The gate requires every exclusion that resolves to a real file to (a) name a tracking issue and
(b) appear in `config/quality/vitest-exclusions.json` with its measured status, so adding one is a
reviewable diff in a dedicated file rather than one more line in a 60-entry array. It deliberately
does not re-run the excluded tests — that costs ~10 minutes and belongs in a periodic job; the
inventory records when each was last measured.

4
package.json
View File

@@ -283,7 +283,9 @@
"release:reconcile": "node scripts/release/reconcile-changelog.mjs",
"test:coverage:runner": "node --max-old-space-size=8192 --import tsx/esm --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test --test-force-exit --test-concurrency=8 tests/unit/*.test.ts \"tests/unit/{api,auth,authz,build,cli,cli-helper,combo,compression,correctness,cors,db,db-adapters,docs,gamification,guardrails,lib,mcp,memory,runtime,security,services,settings,shared,translator,ui,usage}/**/*.test.ts\" \"tests/unit/**/*.test.mjs\" && cross-env DISABLE_SQLITE_AUTO_BACKUP=true NODE_OPTIONS=--max-old-space-size=8192 c8 --merge-async --output-dir coverage --exclude=tests/** --exclude=**/*.test.* --reporter=text-summary --reporter=html --reporter=json-summary --reporter=lcov --check-coverage --statements 60 --lines 60 --functions 60 --branches 60 node --max-old-space-size=8192 --import tsx --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test --test-force-exit --test-concurrency=8 \"tests/unit/dashboard/**/*.test.ts\" && npm run test:unit:serial",
"test:unit:serial": "cross-env DISABLE_SQLITE_AUTO_BACKUP=true node --max-old-space-size=8192 --import tsx/esm --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test --test-force-exit --test-concurrency=1 \"tests/unit/serial/**/*.test.ts\"",
"alibaba:sync-allowlist": "node --import tsx/esm scripts/ops/sync-alibaba-allowlist.mjs"
"alibaba:sync-allowlist": "node --import tsx/esm scripts/ops/sync-alibaba-allowlist.mjs",
"check:vitest-exclusions": "node scripts/check/check-vitest-exclusions.mjs",
"i18n:check-new-keys": "node scripts/i18n/check-new-key-coverage.mjs"
},
"dependencies": {
"@aws-sdk/client-bedrock-runtime": "^3.1120.0",

156
scripts/check/check-vitest-exclusions.mjs Normal file
View File

@@ -0,0 +1,156 @@
#!/usr/bin/env node
/**
* OmniRoute — Vitest exclusion gate (CI gate, blocking).
*
* Every file parked in `vitest.config.ts`'s `exclude` list is a test that does not run.
* A skipped test is indistinguishable from a test that does not exist, with the added
* hazard of LOOKING like coverage to whoever reads the file tree.
*
* Why this gate exists (the incident it encodes): 62 files accumulated behind the comment
* `// #8618 — pre-existing failure; remove this exclusion when fixed`. Issue #8618 was
* CLOSED on 2026-08-11 while the list it tracked kept growing — from 45 entries to 62 —
* each new exclusion inheriting a comment that pointed at a dead issue. When the list was
* finally measured file by file (#13204), **51 of the 62 passed against the current tree
* with no source change**: the exclusions had outlived the failures that justified them by
* months, and nothing in CI could say so.
*
* The gate enforces the two properties that would have caught it:
*
* 1. Every excluded path that resolves to a real file carries an issue reference
* (`#<number>`) in a trailing comment. An exclusion without a tracker is invisible
* debt.
* 2. The set of excluded files matches the checked-in inventory
* (`config/quality/vitest-exclusions.json`). Adding an exclusion becomes a visible,
* reviewable diff in a dedicated file instead of one more line lost in a 60-entry
* array.
*
* What it deliberately does NOT do: re-run the excluded tests to see whether they pass
* again. That costs ~10 minutes and belongs in a periodic job, not in a per-PR gate. The
* inventory records the measured status and the date so a reader knows how stale it is.
*
* Standard tooling exclusions (`node_modules/**`, glob patterns, the live-server E2E specs
* that have their own runner) are exempt — they are configuration, not debt.
*
* Usage:
* node scripts/check/check-vitest-exclusions.mjs # strict, exit 1 on violation
* node scripts/check/check-vitest-exclusions.mjs --json # machine-readable
*/
import fs from "node:fs";
import path from "node:path";
import process from "node:process";
import { fileURLToPath } from "node:url";
const SCRIPT_DIR = path.dirname(fileURLToPath(import.meta.url));
const ROOT = path.resolve(SCRIPT_DIR, "..", "..");
const CONFIG = path.join(ROOT, "vitest.config.ts");
const INVENTORY = path.join(ROOT, "config/quality/vitest-exclusions.json");
/** Exclusions that are tooling configuration rather than parked debt. */
const EXEMPT = new Set([
"node_modules/**",
"dist/**",
"cypress/**",
".idea/**",
".git/**",
".cache/**",
// Live-server E2E: their own runner + vitest.e2e-live.config.ts, never this jsdom job.
"tests/e2e/ecosystem.test.ts",
"tests/e2e/protocol-clients.test.ts",
]);
/**
* Parse the `exclude` array out of vitest.config.ts, keeping each entry's trailing comment.
*
* @returns {Array<{ pattern: string, comment: string }>}
*/
export function parseExclusions(source) {
const block = source.match(/exclude:\s*\[([\s\S]*?)\n {4}\]/);
if (!block) return [];
const out = [];
for (const line of block[1].split("\n")) {
const pattern = line.match(/"([^"]+)"/);
if (!pattern) continue;
const comment = line.slice(line.indexOf(pattern[0]) + pattern[0].length);
out.push({ pattern: pattern[1], comment: comment.trim() });
}
return out;
}
/**
* Pure core: which exclusions violate the gate?
*
* @param {Array<{pattern: string, comment: string}>} entries
* @param {(p: string) => boolean} exists
* @param {string[]} inventory paths recorded in the checked-in inventory
*/
export function findViolations(entries, exists, inventory) {
const tracked = new Set(inventory);
const untracked = [];
const unreferenced = [];
const seen = new Set();
for (const { pattern, comment } of entries) {
if (EXEMPT.has(pattern) || pattern.includes("*")) continue;
if (!exists(pattern)) continue; // a stale path excludes nothing
seen.add(pattern);
if (!/#\d+/.test(comment)) unreferenced.push(pattern);
if (!tracked.has(pattern)) untracked.push(pattern);
}
const orphaned = inventory.filter((p) => !seen.has(p));
return { unreferenced, untracked, orphaned };
}
function main() {
const json = process.argv.includes("--json");
const entries = parseExclusions(fs.readFileSync(CONFIG, "utf8"));
const inventory = fs.existsSync(INVENTORY)
? JSON.parse(fs.readFileSync(INVENTORY, "utf8")).excluded.map((e) => e.file)
: [];
const { unreferenced, untracked, orphaned } = findViolations(
entries,
(p) => fs.existsSync(path.join(ROOT, p)),
inventory
);
if (json) {
console.log(JSON.stringify({ unreferenced, untracked, orphaned }, null, 2));
}
const failed = unreferenced.length + untracked.length + orphaned.length;
if (!failed) {
console.log(
`[vitest-exclusions] OK — ${inventory.length} excluded file(s), each tracked and referenced.`
);
return;
}
if (unreferenced.length) {
console.error(
`\n[vitest-exclusions] FAIL — ${unreferenced.length} exclusion(s) carry no issue reference:`
);
for (const p of unreferenced) console.error(` ✗ ${p}`);
console.error(
" Add a trailing comment naming an OPEN tracking issue, e.g. // #13204 — reason"
);
}
if (untracked.length) {
console.error(
`\n[vitest-exclusions] FAIL — ${untracked.length} exclusion(s) missing from ${path.relative(ROOT, INVENTORY)}:`
);
for (const p of untracked) console.error(` ✗ ${p}`);
console.error(" Record it there with its measured status, so the debt is reviewable.");
}
if (orphaned.length) {
console.error(
`\n[vitest-exclusions] FAIL — ${orphaned.length} inventory entr(ies) no longer excluded:`
);
for (const p of orphaned) console.error(` ✗ ${p}`);
console.error(" The test runs again — drop it from the inventory.");
}
process.exit(1);
}
if (import.meta.url === `file://${process.argv[1]}`) main();

201
scripts/i18n/check-new-key-coverage.mjs Normal file
View File

@@ -0,0 +1,201 @@
#!/usr/bin/env node
/**
* OmniRoute — NEW-key i18n coverage gate (CI gate, blocking).
*
* Sibling of `check-ui-value-drift.mjs`. That one catches an English value that was
* REWRITTEN while translations were left behind; this one catches an English key that was
* ADDED while some locales never received it.
*
* Why no existing gate sees this (the incident it encodes): Phase 3 of the Orchestration
* Canvas added eleven keys and translated them across the 42 locales that existed at the
* time. Hours later the EU-language batch (#13044) took the repo to 51 locales. The nine
* new files — el, et, ga, hr, lt, lv, mt, sl, sr — never received those eleven keys, so the
* compare-runs panel rendered in English for those users.
*
* `check-ui-keys-coverage.mjs` could not catch it: it enforces an 80% floor PER LOCALE, and
* eleven absent keys out of ~13,000 leaves coverage at 99.9%. A percentage per language
* cannot express "this feature shipped untranslated" — an entire feature can land in a new
* locale with no text and never move the number.
*
* `deepMergeFallback` (src/i18n/request.ts) does substitute English for an absent key, so
* the failure mode is untranslated UI rather than blank UI. That is a real defect, not a
* cosmetic one, and it is silent by construction.
*
* How this gate works: DIFF-AWARE, like its sibling. It compares the English catalog at the
* merge base against the working tree; every key that is NEW in English must be present and
* non-placeholder in every locale. Pre-existing gaps are deliberately frozen — this gate
* judges only what the current change adds, so it can be turned on without a migration.
*
* Escape hatch, same as the sibling: set the value to `__MISSING__:<english>` to make the
* runtime fall back to correct English and queue the key for the translation pipeline.
* NOTE that `vi` bans placeholders (tests/unit/i18n-vi-completeness.test.ts), so `vi` needs
* a real translation.
*
* Usage:
* node scripts/i18n/check-new-key-coverage.mjs # strict, exit 1
* node scripts/i18n/check-new-key-coverage.mjs --warn # report, exit 0
* node scripts/i18n/check-new-key-coverage.mjs --json
* BASE_REF=origin/release/vX.Y.Z node scripts/i18n/check-new-key-coverage.mjs
*
* Graceful SKIP (exit 0) when the base catalog cannot be resolved — shallow clone, or a
* brand-new catalog. Mirrors the SKIP in check-ui-value-drift.mjs.
*/
import fs from "node:fs";
import path from "node:path";
import process from "node:process";
import { execFileSync } from "node:child_process";
import { fileURLToPath } from "node:url";
const SCRIPT_DIR = path.dirname(fileURLToPath(import.meta.url));
const ROOT = path.resolve(SCRIPT_DIR, "..", "..");
const MESSAGES_REL = "src/i18n/messages";
const PLACEHOLDER_PREFIX = "__MISSING__:";
/** Flatten a nested catalog into `{ "a.b.c": value }`. */
export function flattenLeaves(node, prefix = "", out = {}) {
for (const [key, value] of Object.entries(node ?? {})) {
const dotted = prefix ? `${prefix}.${key}` : key;
if (value !== null && typeof value === "object" && !Array.isArray(value)) {
flattenLeaves(value, dotted, out);
} else {
out[dotted] = value;
}
}
return out;
}
/**
* Pure core: which (key, locale) pairs are keys new in English that a locale never got?
*
* A `__MISSING__:` placeholder counts as satisfied — it is the documented, runtime-correct
* way to defer a translation.
*
* @param {object} args
* @param {object} args.baseEn en.json at the base ref
* @param {object} args.headEn en.json in the working tree
* @param {Record<string, object>} args.headLocales locale -> catalog in the working tree
* @returns {Array<{ key: string, locale: string }>} sorted, stable
*/
export function findUntranslatedNewKeys({ baseEn, headEn, headLocales }) {
const base = flattenLeaves(baseEn);
const head = flattenLeaves(headEn);
const newKeys = Object.keys(head).filter(
(k) => !(k in base) && typeof head[k] === "string" && head[k].trim() !== ""
);
if (!newKeys.length) return [];
const gaps = [];
for (const [locale, catalog] of Object.entries(headLocales)) {
const flat = flattenLeaves(catalog);
for (const key of newKeys) {
const value = flat[key];
const satisfied =
typeof value === "string" && (value.trim() !== "" || value.startsWith(PLACEHOLDER_PREFIX));
if (!satisfied) gaps.push({ key, locale });
}
}
gaps.sort((a, b) => a.key.localeCompare(b.key) || a.locale.localeCompare(b.locale));
return gaps;
}
function git(args) {
return execFileSync("git", args, {
cwd: ROOT,
encoding: "utf8",
stdio: ["ignore", "pipe", "pipe"],
});
}
function readPackageVersion() {
try {
return JSON.parse(fs.readFileSync(path.join(ROOT, "package.json"), "utf8")).version;
} catch {
return null;
}
}
function defaultBaseRef() {
const v = readPackageVersion();
return v && /^\d+\.\d+\.\d+$/.test(v) ? `origin/release/v${v}` : null;
}
function resolveDiffBase(baseRef) {
try {
return git(["merge-base", "HEAD", baseRef]).trim();
} catch {
return baseRef;
}
}
function readCatalogAtRef(ref, relPath) {
try {
return JSON.parse(git(["show", `${ref}:${relPath}`]));
} catch {
return null;
}
}
function main() {
const argv = process.argv.slice(2);
const opts = { json: argv.includes("--json"), warn: argv.includes("--warn") };
const baseRef = process.env.BASE_REF || defaultBaseRef();
const skip = (reason) => {
if (opts.json) process.stdout.write(JSON.stringify({ ok: true, skipped: true, reason }) + "\n");
else console.log(`[i18n-new-keys] SKIP reason=${reason}`);
process.exit(0);
};
if (!baseRef) skip("base-unresolved");
const base = resolveDiffBase(baseRef);
const baseEn = readCatalogAtRef(base, `${MESSAGES_REL}/en.json`);
if (!baseEn) skip("base-catalog-unreadable");
const dir = path.join(ROOT, MESSAGES_REL);
const headEn = JSON.parse(fs.readFileSync(path.join(dir, "en.json"), "utf8"));
const headLocales = {};
for (const file of fs.readdirSync(dir)) {
if (!file.endsWith(".json") || file === "en.json") continue;
try {
headLocales[file.replace(/\.json$/, "")] = JSON.parse(
fs.readFileSync(path.join(dir, file), "utf8")
);
} catch {
/* a malformed catalog is another gate's problem */
}
}
const gaps = findUntranslatedNewKeys({ baseEn, headEn, headLocales });
if (opts.json) {
console.log(JSON.stringify({ ok: gaps.length === 0, gaps }, null, 2));
}
if (!gaps.length) {
console.log(
`[i18n-new-keys] PASS — every key new in English reached all ${Object.keys(headLocales).length} locale(s).`
);
return;
}
const byKey = new Map();
for (const g of gaps) {
if (!byKey.has(g.key)) byKey.set(g.key, []);
byKey.get(g.key).push(g.locale);
}
const label = opts.warn ? "WARN" : "FAIL";
console.error(
`\n[i18n-new-keys] ${label} — ${byKey.size} new English key(s) missing from some locales:`
);
for (const [key, locales] of byKey) {
console.error(` ✗ ${key} — missing in ${locales.length}: ${locales.join(", ")}`);
}
console.error(
"\n Translate them, or set `__MISSING__:<english>` to defer (the runtime then falls back\n" +
" to English). `vi` bans placeholders — it needs a real translation."
);
if (!opts.warn) process.exit(1);
}
if (import.meta.url === `file://${process.argv[1]}`) main();

67
tests/unit/check-vitest-exclusions.test.ts Normal file
View File

@@ -0,0 +1,67 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import fs from "node:fs";
import path from "node:path";
import { parseExclusions, findViolations } from "../../scripts/check/check-vitest-exclusions.mjs";
const ROOT = path.resolve(import.meta.dirname, "..", "..");
/** Every excluded file in the inventory is a real path — a stale entry excludes nothing. */
test("the checked-in inventory only lists files that exist", () => {
const inv = JSON.parse(
fs.readFileSync(path.join(ROOT, "config/quality/vitest-exclusions.json"), "utf8")
);
assert.ok(inv.excluded.length > 0, "inventory is not empty");
for (const entry of inv.excluded) {
assert.ok(fs.existsSync(path.join(ROOT, entry.file)), `${entry.file} exists`);
assert.match(entry.issue, /^#\d+$/, `${entry.file} names a tracking issue`);
}
});
test("the live config satisfies the gate", () => {
const entries = parseExclusions(fs.readFileSync(path.join(ROOT, "vitest.config.ts"), "utf8"));
const inv = JSON.parse(
fs.readFileSync(path.join(ROOT, "config/quality/vitest-exclusions.json"), "utf8")
);
const v = findViolations(
entries,
(p) => fs.existsSync(path.join(ROOT, p)),
inv.excluded.map((e: { file: string }) => e.file)
);
assert.deepEqual(v, { unreferenced: [], untracked: [], orphaned: [] });
});
test("parseExclusions keeps each entry's trailing comment", () => {
const src = ` test: {\n exclude: [\n "node_modules/**",\n "tests/a.test.ts", // #123 — reason\n ]\n }`;
assert.deepEqual(parseExclusions(src), [
{ pattern: "node_modules/**", comment: "," },
{ pattern: "tests/a.test.ts", comment: ", // #123 — reason" },
]);
});
test("an exclusion with no issue reference fails the gate", () => {
const entries = [{ pattern: "tests/a.test.ts", comment: ", // just because" }];
const v = findViolations(entries, () => true, ["tests/a.test.ts"]);
assert.deepEqual(v.unreferenced, ["tests/a.test.ts"]);
});
test("an exclusion absent from the inventory fails the gate", () => {
const entries = [{ pattern: "tests/a.test.ts", comment: ", // #123" }];
const v = findViolations(entries, () => true, []);
assert.deepEqual(v.untracked, ["tests/a.test.ts"]);
});
test("an inventory entry that is no longer excluded fails the gate", () => {
const v = findViolations([], () => true, ["tests/revived.test.ts"]);
assert.deepEqual(v.orphaned, ["tests/revived.test.ts"]);
});
test("tooling exclusions and globs are exempt, and a stale path is ignored", () => {
const entries = [
{ pattern: "node_modules/**", comment: "," },
{ pattern: "tests/unit/**/*.test.tsx", comment: "," },
{ pattern: "tests/deleted.test.ts", comment: "," },
];
const v = findViolations(entries, (p) => p !== "tests/deleted.test.ts", []);
assert.deepEqual(v, { unreferenced: [], untracked: [], orphaned: [] });
});

78
tests/unit/i18n-new-key-coverage.test.ts Normal file
View File

@@ -0,0 +1,78 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { findUntranslatedNewKeys } from "../../scripts/i18n/check-new-key-coverage.mjs";
const en = (extra: Record<string, string> = {}) => ({ ui: { existing: "Existing", ...extra } });
test("a key that already existed is never flagged, however bad its translations", () => {
const gaps = findUntranslatedNewKeys({
baseEn: en(),
headEn: en(),
headLocales: { pt: { ui: {} } },
});
assert.deepEqual(gaps, [], "pre-existing debt stays frozen — this gate judges only new keys");
});
test("a new English key missing from a locale is flagged", () => {
const gaps = findUntranslatedNewKeys({
baseEn: en(),
headEn: en({ fresh: "Fresh" }),
headLocales: { pt: { ui: { existing: "Existente" } }, de: { ui: { existing: "Vorhanden" } } },
});
assert.deepEqual(gaps, [
{ key: "ui.fresh", locale: "de" },
{ key: "ui.fresh", locale: "pt" },
]);
});
test("a new key translated everywhere passes", () => {
const gaps = findUntranslatedNewKeys({
baseEn: en(),
headEn: en({ fresh: "Fresh" }),
headLocales: { pt: { ui: { existing: "Existente", fresh: "Novo" } } },
});
assert.deepEqual(gaps, []);
});
test("a __MISSING__ placeholder satisfies the gate — it is the documented deferral", () => {
const gaps = findUntranslatedNewKeys({
baseEn: en(),
headEn: en({ fresh: "Fresh" }),
headLocales: { pt: { ui: { existing: "Existente", fresh: "__MISSING__:Fresh" } } },
});
assert.deepEqual(gaps, []);
});
test("an empty string does NOT satisfy the gate", () => {
const gaps = findUntranslatedNewKeys({
baseEn: en(),
headEn: en({ fresh: "Fresh" }),
headLocales: { pt: { ui: { existing: "Existente", fresh: " " } } },
});
assert.deepEqual(gaps, [{ key: "ui.fresh", locale: "pt" }]);
});
/**
* The incident this gate encodes: Phase 3 added keys against 42 locales; the EU batch then
* took the repo to 51, and the nine newcomers never received them.
*/
test("a locale added AFTER the key still has to carry it", () => {
const gaps = findUntranslatedNewKeys({
baseEn: en(),
headEn: en({ compareMode: "Compare runs" }),
headLocales: {
pt: { ui: { existing: "Existente", compareMode: "Comparar execuções" } },
el: { ui: { existing: "Υπάρχον" } },
},
});
assert.deepEqual(gaps, [{ key: "ui.compareMode", locale: "el" }]);
});
test("a new key whose English value is empty is not enforced", () => {
const gaps = findUntranslatedNewKeys({
baseEn: en(),
headEn: en({ blank: "" }),
headLocales: { pt: { ui: { existing: "Existente" } } },
});
assert.deepEqual(gaps, []);
});