Compare commits

...

4 Commits

Author SHA1 Message Date
diegosouzapw
1f78aef4d3 Merge release/v3.8.51 into chore/revive-excluded-vitest-tests
# Conflicts:
#	docs/architecture/QUALITY_GATES.md
2026-09-14 13:33:55 -03:00
diegosouzapw
b4699bd0d4 Merge remote-tracking branch 'origin/release/v3.8.51' into chore/revive-excluded-vitest-tests 2026-09-10 18:26:00 -03:00
diegosouzapw
4a4545b662 chore(ci): gate the two blind spots that let silent debt accumulate
check:vitest-exclusions — a file in vitest.config.ts's exclude list is a test that
does not run, indistinguishable from one that does not exist except that it looks
like coverage. 62 files accumulated behind '// #8618 — pre-existing failure',
while #8618 itself was closed on 2026-08-11 and the list grew from 45 to 62. The
gate requires every exclusion to name a tracker and to appear in
config/quality/vitest-exclusions.json with its measured status, so growth is a
reviewable diff in a dedicated file. It does not re-run the tests — that is a
periodic job, and the inventory records when each was last measured.

check-new-key-coverage — sibling of check-ui-value-drift. That one catches a
rewritten English value leaving stale translations; this one catches a new English
key that some locales never received. check-ui-keys-coverage cannot: it is a
percentage floor per locale, and 11 absent keys out of ~13,000 leaves it at 99.9%.
Proven retroactively against the real incident — with BASE_REF set before the
Phase 3 merge it flags all 11 canvas keys across exactly the nine EU locales that
missed them, plus 3 keys from other features with the same problem.

Both are diff-aware against the merge base, so pre-existing gaps stay frozen and
neither needed a migration. Both wired into ci.yml and documented in
docs/architecture/QUALITY_GATES.md, with 14 tests covering the pure cores.

Refs #13204
2026-09-10 18:25:24 -03:00
diegosouzapw
e3493c6de2 chore(tests): revive 51 vitest files parked behind a closed tracker
Of the 62 files excluded from Vitest with the comment
'// #8618 — pre-existing failure; remove this exclusion when fixed', 51 pass
against the current tree with no source change: the exclusions outlived the
failures that justified them. #8618 was closed on 2026-08-11 while the list it
tracked grew from 45 to 62 entries, so every later exclusion inherited a comment
pointing at a dead issue and nobody revisited any of them.

Each file was measured on its own before being re-enabled, and the full suite was
run before and after. The 51 revived files do not appear in the failure list of
either run.

The 11 that still fail keep their exclusion and now reference #13204, the live
tracker, so the next reader is pointed somewhere real.

Known trade-off, stated rather than hidden: the blocking suite grows from 325 to
376 files (+372 tests, ~17%). That extra parallel load pressures the tests that
already fail intermittently under a loaded full-suite run on the base
(compression*, lobe-provider-icons-stepfun, resilience-connections,
orchestrationHistoryTab). Those pass 25/25 in isolation with this config applied,
so this change does not break them logically — it raises the load that already
tips them over. They belong to the base-red family tracked by #12732.

Refs #13204
2026-09-10 12:39:56 -03:00
11 changed files with 657 additions and 63 deletions

View File

@@ -144,6 +144,12 @@ jobs:
- run: npm run check:test-discovery
- run: npm run check:radar-sentinels
- run: npm run check:tracked-artifacts
# A test parked in vitest.config.ts's exclude list does not run, and looks like
# coverage to whoever reads the tree. 62 files accumulated behind a comment pointing
# at #8618 — closed in August while the list grew to 62; 51 of them passed when
# finally measured (#13204). This gate requires every exclusion to name a tracker and
# to appear in config/quality/vitest-exclusions.json, so the debt stays reviewable.
- run: npm run check:vitest-exclusions
# (gap 30) Also lives in quality.yml's PR-only "Merge integrity" job — because the
# CHANGELOG half of that job needs a base to diff against. This half does NOT: the
# generator either reproduces the committed SKILL.md files or it does not.
@@ -515,6 +521,16 @@ jobs:
env:
BASE_REF: ${{ github.base_ref && format('origin/{0}', github.base_ref) || '' }}
run: node scripts/i18n/check-ui-value-drift.mjs
# Sibling of the drift gate above. That one catches an English value that was
# REWRITTEN; this one catches an English key that was ADDED while some locales never
# got it. The coverage gate at the top of this job cannot: it is a percentage per
# locale, and 11 absent keys out of ~13,000 leaves coverage at 99.9%. Incident: the
# Phase 3 canvas keys were translated across the 42 locales that existed, then the EU
# batch (#13044) took the repo to 51 and the nine newcomers shipped untranslated.
- name: i18n new-key coverage (a new key must reach every locale)
env:
BASE_REF: ${{ github.base_ref && format('origin/{0}', github.base_ref) || '' }}
run: node scripts/i18n/check-new-key-coverage.mjs
# #8038: cheap glossary/protected-terms consistency gate —
# complements i18n-ui-coverage (key parity) and the ICU `i18n` job below

View File

@@ -0,0 +1,8 @@
- **chore(ci):** two gates that close the blind spots behind the exclusions above.
`check:vitest-exclusions` requires every Vitest exclusion to name a tracking issue and to
appear in `config/quality/vitest-exclusions.json` — the previous list grew to 62 files behind
a comment pointing at an issue that had been closed for a month. `check-new-key-coverage`
requires a key newly added to `en.json` to reach every locale; the existing coverage gate is a
percentage floor per locale, so eleven absent keys out of ~13,000 left it at 99.9% while a
whole feature shipped untranslated in nine languages. Both are diff-aware, so pre-existing
debt stays frozen and neither needed a migration to turn on.

View File

@@ -0,0 +1,6 @@
- **chore(tests):** 51 test files that had been excluded from Vitest are running again, restoring
roughly 350 assertions to the blocking suite. Each was measured individually first: of the 62
files parked behind the `// #8618 — pre-existing failure` comment, 51 pass against the current
tree with no source change, so the exclusions had outlived the failures they were added for.
The 11 that genuinely still fail stay excluded, but now point at a live tracker (#13204) rather
than at #8618, which was closed in August while the list it tracked kept growing.

View File

@@ -0,0 +1,72 @@
{
"_comment": "Inventário dos arquivos de teste excluídos do Vitest. Toda entrada precisa de uma issue de rastreio ABERTA. Gate: npm run check:vitest-exclusions. Contexto: #13204.",
"_measured": "2026-09-10 — cada arquivo rodado isoladamente com as exclusões removidas",
"excluded": [
{
"file": "tests/unit/ui/request-logger-autorefresh-visibility-3972.test.tsx",
"issue": "#13204",
"measured": "2026-09-10",
"status": "Tests 1 failed | 5 passed (6)"
},
{
"file": "src/app/(dashboard)/dashboard/webhooks/__tests__/webhook-wizard.test.tsx",
"issue": "#13204",
"measured": "2026-09-10",
"status": "Tests 1 failed | 6 passed (7)"
},
{
"file": "tests/unit/ui/logs-page-detail-modal-reopen-on-close.test.tsx",
"issue": "#13204",
"measured": "2026-09-10",
"status": "Tests 2 failed (2)"
},
{
"file": "tests/unit/ui/agent-card.test.tsx",
"issue": "#13204",
"measured": "2026-09-10",
"status": "Tests 3 failed | 1 passed (4)"
},
{
"file": "src/app/(dashboard)/dashboard/endpoint/__tests__/ApiEndpointsTab.test.tsx",
"issue": "#13204",
"measured": "2026-09-10",
"status": "Tests 2 failed | 1 passed (3)"
},
{
"file": "src/app/(dashboard)/dashboard/cache/__tests__/CacheTrends.test.tsx",
"issue": "#13204",
"measured": "2026-09-10",
"status": "Tests 7 failed | 6 passed (13)"
},
{
"file": "src/app/(dashboard)/dashboard/cache/__tests__/IdempotencyLayer.test.tsx",
"issue": "#13204",
"measured": "2026-09-10",
"status": "Tests 9 failed | 4 passed (13)"
},
{
"file": "src/app/(dashboard)/dashboard/cache/__tests__/CachePerformance.test.tsx",
"issue": "#13204",
"measured": "2026-09-10",
"status": "Tests 10 failed | 4 passed (14)"
},
{
"file": "src/app/(dashboard)/dashboard/discovery/__tests__/DiscoveryPageClient.test.tsx",
"issue": "#13204",
"measured": "2026-09-10",
"status": "Tests 2 failed | 1 passed (3)"
},
{
"file": "tests/unit/ui/combos-page-smoke.test.tsx",
"issue": "#13204",
"measured": "2026-09-10",
"status": "Tests 1 failed (1)"
},
{
"file": "tests/unit/ui/evals-tab-smoke.test.tsx",
"issue": "#13204",
"measured": "2026-09-10",
"status": "Tests 1 failed (1)"
}
]
}

View File

@@ -71,6 +71,7 @@ Runs on every PR to `main`. Blocks merge on failure.
| `check:lockfile` | `package-lock.json` integrity — https registry, integrity hashes, no host overrides | Yes |
| `check:licenses` | SPDX license allowlist for production dependencies | Yes |
| `check:tracked-artifacts` | No build artifacts / committed `node_modules` symlinks (also runs in husky pre-commit; pre-push is intentionally light — #6716) | Yes |
| `check:vitest-exclusions` | Every Vitest exclusion names a tracking issue and appears in `config/quality/vitest-exclusions.json` (#13204) | Yes |
| `check:file-size` | No source file exceeds the per-extension cap (ratchet: frozen large files in `frozen` list) | Yes |
| `check:error-helper` | Error responses in executors/handlers use `buildErrorBody()` / `sanitizeErrorMessage()` (Hard Rule #12) | Yes |
| `check:migration-numbering` | Migration SQL files are sequentially numbered, no gaps or duplicates | Yes |
@@ -143,6 +144,7 @@ Runs on every PR to `main`. Blocks merge on failure.
| --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------ |
| `check-ui-keys-coverage` (inline) | UI i18n key coverage is ≥ 65% | Yes |
| `check-ui-value-drift` (inline) | A rewritten English **value** leaves no stale translation behind | Yes |
| `check-new-key-coverage` (inline) | A **new** English key reaches every locale | Yes |
| `check-translation-ratio` | Real-translation ratio per locale (identical-to-English / placeholder / missing leaves outside the allowlist) must not exceed `config/quality/i18n-translation-baseline.json` + slack | **Advisory** |
Needs `fetch-depth: 0` — the value-drift gate diffs `en.json` against the merge base.
@@ -533,3 +535,40 @@ several "obvious" merges turned out to hide debt and are **not** clean drop-ins.
## Related Documentation
- Supply-chain (provenance, SBOM, Trivy, Scorecard): [`docs/security/SUPPLY_CHAIN.md`](../security/SUPPLY_CHAIN.md)
#### `check-new-key-coverage` — new-key i18n gate
Sibling of `check-ui-value-drift`. That one catches an English value that was **rewritten**
while its translations were left behind; this one catches an English key that was **added**
while some locales never received it.
`check-ui-keys-coverage` cannot see this class: it enforces a percentage floor per locale, and
eleven absent keys out of ~13,000 leaves coverage at 99.9%. A percentage per language cannot
express "this feature shipped untranslated" — an entire feature can land in a new locale with no
text and never move the number.
The incident it encodes: Phase 3 of the Orchestration Canvas translated its eleven keys across
the 42 locales that existed at the time. Hours later the EU-language batch (#13044) took the repo
to 51 locales, and the nine newcomers (`el`, `et`, `ga`, `hr`, `lt`, `lv`, `mt`, `sl`, `sr`) never
received them. `deepMergeFallback` substitutes English for an absent key, so the failure mode was
untranslated UI rather than blank UI — real, and silent by construction.
Like its sibling it is **diff-aware**, comparing English at the merge base against the working
tree, so pre-existing gaps stay frozen and the gate needed no migration to turn on. Escape hatch:
`__MISSING__:<english>` defers a translation while keeping the runtime correct. `vi` bans
placeholders (`tests/unit/i18n-vi-completeness.test.ts`) and needs a real translation.
#### `check-vitest-exclusions` — parked-test gate
A file in `vitest.config.ts`'s `exclude` list is a test that does not run, and it looks like
coverage to whoever reads the tree. Sixty-two files accumulated behind the comment
`// #8618 — pre-existing failure; remove this exclusion when fixed`. Issue #8618 was closed on
2026-08-11 while the list it tracked grew from 45 entries to 62, each new one inheriting a comment
pointing at a dead issue. When the list was finally measured file by file (#13204), **51 of the 62
passed against the current tree with no source change**.
The gate requires every exclusion that resolves to a real file to (a) name a tracking issue and
(b) appear in `config/quality/vitest-exclusions.json` with its measured status, so adding one is a
reviewable diff in a dedicated file rather than one more line in a 60-entry array. It deliberately
does not re-run the excluded tests — that costs ~10 minutes and belongs in a periodic job; the
inventory records when each was last measured.

View File

@@ -283,7 +283,9 @@
"release:reconcile": "node scripts/release/reconcile-changelog.mjs",
"test:coverage:runner": "node --max-old-space-size=8192 --import tsx/esm --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test --test-force-exit --test-concurrency=8 tests/unit/*.test.ts \"tests/unit/{api,auth,authz,build,cli,cli-helper,combo,compression,correctness,cors,db,db-adapters,docs,gamification,guardrails,lib,mcp,memory,runtime,security,services,settings,shared,translator,ui,usage}/**/*.test.ts\" \"tests/unit/**/*.test.mjs\" && cross-env DISABLE_SQLITE_AUTO_BACKUP=true NODE_OPTIONS=--max-old-space-size=8192 c8 --merge-async --output-dir coverage --exclude=tests/** --exclude=**/*.test.* --reporter=text-summary --reporter=html --reporter=json-summary --reporter=lcov --check-coverage --statements 60 --lines 60 --functions 60 --branches 60 node --max-old-space-size=8192 --import tsx --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test --test-force-exit --test-concurrency=8 \"tests/unit/dashboard/**/*.test.ts\" && npm run test:unit:serial",
"test:unit:serial": "cross-env DISABLE_SQLITE_AUTO_BACKUP=true node --max-old-space-size=8192 --import tsx/esm --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test --test-force-exit --test-concurrency=1 \"tests/unit/serial/**/*.test.ts\"",
"alibaba:sync-allowlist": "node --import tsx/esm scripts/ops/sync-alibaba-allowlist.mjs"
"alibaba:sync-allowlist": "node --import tsx/esm scripts/ops/sync-alibaba-allowlist.mjs",
"check:vitest-exclusions": "node scripts/check/check-vitest-exclusions.mjs",
"i18n:check-new-keys": "node scripts/i18n/check-new-key-coverage.mjs"
},
"dependencies": {
"@aws-sdk/client-bedrock-runtime": "^3.1120.0",

View File

@@ -0,0 +1,156 @@
#!/usr/bin/env node
/**
* OmniRoute — Vitest exclusion gate (CI gate, blocking).
*
* Every file parked in `vitest.config.ts`'s `exclude` list is a test that does not run.
* A skipped test is indistinguishable from a test that does not exist, with the added
* hazard of LOOKING like coverage to whoever reads the file tree.
*
* Why this gate exists (the incident it encodes): 62 files accumulated behind the comment
* `// #8618 — pre-existing failure; remove this exclusion when fixed`. Issue #8618 was
* CLOSED on 2026-08-11 while the list it tracked kept growing — from 45 entries to 62 —
* each new exclusion inheriting a comment that pointed at a dead issue. When the list was
* finally measured file by file (#13204), **51 of the 62 passed against the current tree
* with no source change**: the exclusions had outlived the failures that justified them by
* months, and nothing in CI could say so.
*
* The gate enforces the two properties that would have caught it:
*
* 1. Every excluded path that resolves to a real file carries an issue reference
* (`#<number>`) in a trailing comment. An exclusion without a tracker is invisible
* debt.
* 2. The set of excluded files matches the checked-in inventory
* (`config/quality/vitest-exclusions.json`). Adding an exclusion becomes a visible,
* reviewable diff in a dedicated file instead of one more line lost in a 60-entry
* array.
*
* What it deliberately does NOT do: re-run the excluded tests to see whether they pass
* again. That costs ~10 minutes and belongs in a periodic job, not in a per-PR gate. The
* inventory records the measured status and the date so a reader knows how stale it is.
*
* Standard tooling exclusions (`node_modules/**`, glob patterns, the live-server E2E specs
* that have their own runner) are exempt — they are configuration, not debt.
*
* Usage:
* node scripts/check/check-vitest-exclusions.mjs # strict, exit 1 on violation
* node scripts/check/check-vitest-exclusions.mjs --json # machine-readable
*/
import fs from "node:fs";
import path from "node:path";
import process from "node:process";
import { fileURLToPath } from "node:url";
const SCRIPT_DIR = path.dirname(fileURLToPath(import.meta.url));
const ROOT = path.resolve(SCRIPT_DIR, "..", "..");
const CONFIG = path.join(ROOT, "vitest.config.ts");
const INVENTORY = path.join(ROOT, "config/quality/vitest-exclusions.json");
/** Exclusions that are tooling configuration rather than parked debt. */
const EXEMPT = new Set([
"node_modules/**",
"dist/**",
"cypress/**",
".idea/**",
".git/**",
".cache/**",
// Live-server E2E: their own runner + vitest.e2e-live.config.ts, never this jsdom job.
"tests/e2e/ecosystem.test.ts",
"tests/e2e/protocol-clients.test.ts",
]);
/**
* Parse the `exclude` array out of vitest.config.ts, keeping each entry's trailing comment.
*
* @returns {Array<{ pattern: string, comment: string }>}
*/
export function parseExclusions(source) {
const block = source.match(/exclude:\s*\[([\s\S]*?)\n {4}\]/);
if (!block) return [];
const out = [];
for (const line of block[1].split("\n")) {
const pattern = line.match(/"([^"]+)"/);
if (!pattern) continue;
const comment = line.slice(line.indexOf(pattern[0]) + pattern[0].length);
out.push({ pattern: pattern[1], comment: comment.trim() });
}
return out;
}
/**
* Pure core: which exclusions violate the gate?
*
* @param {Array<{pattern: string, comment: string}>} entries
* @param {(p: string) => boolean} exists
* @param {string[]} inventory paths recorded in the checked-in inventory
*/
export function findViolations(entries, exists, inventory) {
const tracked = new Set(inventory);
const untracked = [];
const unreferenced = [];
const seen = new Set();
for (const { pattern, comment } of entries) {
if (EXEMPT.has(pattern) || pattern.includes("*")) continue;
if (!exists(pattern)) continue; // a stale path excludes nothing
seen.add(pattern);
if (!/#\d+/.test(comment)) unreferenced.push(pattern);
if (!tracked.has(pattern)) untracked.push(pattern);
}
const orphaned = inventory.filter((p) => !seen.has(p));
return { unreferenced, untracked, orphaned };
}
function main() {
const json = process.argv.includes("--json");
const entries = parseExclusions(fs.readFileSync(CONFIG, "utf8"));
const inventory = fs.existsSync(INVENTORY)
? JSON.parse(fs.readFileSync(INVENTORY, "utf8")).excluded.map((e) => e.file)
: [];
const { unreferenced, untracked, orphaned } = findViolations(
entries,
(p) => fs.existsSync(path.join(ROOT, p)),
inventory
);
if (json) {
console.log(JSON.stringify({ unreferenced, untracked, orphaned }, null, 2));
}
const failed = unreferenced.length + untracked.length + orphaned.length;
if (!failed) {
console.log(
`[vitest-exclusions] OK — ${inventory.length} excluded file(s), each tracked and referenced.`
);
return;
}
if (unreferenced.length) {
console.error(
`\n[vitest-exclusions] FAIL — ${unreferenced.length} exclusion(s) carry no issue reference:`
);
for (const p of unreferenced) console.error(`${p}`);
console.error(
" Add a trailing comment naming an OPEN tracking issue, e.g. // #13204 — reason"
);
}
if (untracked.length) {
console.error(
`\n[vitest-exclusions] FAIL — ${untracked.length} exclusion(s) missing from ${path.relative(ROOT, INVENTORY)}:`
);
for (const p of untracked) console.error(`${p}`);
console.error(" Record it there with its measured status, so the debt is reviewable.");
}
if (orphaned.length) {
console.error(
`\n[vitest-exclusions] FAIL — ${orphaned.length} inventory entr(ies) no longer excluded:`
);
for (const p of orphaned) console.error(`${p}`);
console.error(" The test runs again — drop it from the inventory.");
}
process.exit(1);
}
if (import.meta.url === `file://${process.argv[1]}`) main();

View File

@@ -0,0 +1,201 @@
#!/usr/bin/env node
/**
* OmniRoute — NEW-key i18n coverage gate (CI gate, blocking).
*
* Sibling of `check-ui-value-drift.mjs`. That one catches an English value that was
* REWRITTEN while translations were left behind; this one catches an English key that was
* ADDED while some locales never received it.
*
* Why no existing gate sees this (the incident it encodes): Phase 3 of the Orchestration
* Canvas added eleven keys and translated them across the 42 locales that existed at the
* time. Hours later the EU-language batch (#13044) took the repo to 51 locales. The nine
* new files — el, et, ga, hr, lt, lv, mt, sl, sr — never received those eleven keys, so the
* compare-runs panel rendered in English for those users.
*
* `check-ui-keys-coverage.mjs` could not catch it: it enforces an 80% floor PER LOCALE, and
* eleven absent keys out of ~13,000 leaves coverage at 99.9%. A percentage per language
* cannot express "this feature shipped untranslated" — an entire feature can land in a new
* locale with no text and never move the number.
*
* `deepMergeFallback` (src/i18n/request.ts) does substitute English for an absent key, so
* the failure mode is untranslated UI rather than blank UI. That is a real defect, not a
* cosmetic one, and it is silent by construction.
*
* How this gate works: DIFF-AWARE, like its sibling. It compares the English catalog at the
* merge base against the working tree; every key that is NEW in English must be present and
* non-placeholder in every locale. Pre-existing gaps are deliberately frozen — this gate
* judges only what the current change adds, so it can be turned on without a migration.
*
* Escape hatch, same as the sibling: set the value to `__MISSING__:<english>` to make the
* runtime fall back to correct English and queue the key for the translation pipeline.
* NOTE that `vi` bans placeholders (tests/unit/i18n-vi-completeness.test.ts), so `vi` needs
* a real translation.
*
* Usage:
* node scripts/i18n/check-new-key-coverage.mjs # strict, exit 1
* node scripts/i18n/check-new-key-coverage.mjs --warn # report, exit 0
* node scripts/i18n/check-new-key-coverage.mjs --json
* BASE_REF=origin/release/vX.Y.Z node scripts/i18n/check-new-key-coverage.mjs
*
* Graceful SKIP (exit 0) when the base catalog cannot be resolved — shallow clone, or a
* brand-new catalog. Mirrors the SKIP in check-ui-value-drift.mjs.
*/
import fs from "node:fs";
import path from "node:path";
import process from "node:process";
import { execFileSync } from "node:child_process";
import { fileURLToPath } from "node:url";
const SCRIPT_DIR = path.dirname(fileURLToPath(import.meta.url));
const ROOT = path.resolve(SCRIPT_DIR, "..", "..");
const MESSAGES_REL = "src/i18n/messages";
const PLACEHOLDER_PREFIX = "__MISSING__:";
/** Flatten a nested catalog into `{ "a.b.c": value }`. */
export function flattenLeaves(node, prefix = "", out = {}) {
for (const [key, value] of Object.entries(node ?? {})) {
const dotted = prefix ? `${prefix}.${key}` : key;
if (value !== null && typeof value === "object" && !Array.isArray(value)) {
flattenLeaves(value, dotted, out);
} else {
out[dotted] = value;
}
}
return out;
}
/**
* Pure core: which (key, locale) pairs are keys new in English that a locale never got?
*
* A `__MISSING__:` placeholder counts as satisfied — it is the documented, runtime-correct
* way to defer a translation.
*
* @param {object} args
* @param {object} args.baseEn en.json at the base ref
* @param {object} args.headEn en.json in the working tree
* @param {Record<string, object>} args.headLocales locale -> catalog in the working tree
* @returns {Array<{ key: string, locale: string }>} sorted, stable
*/
export function findUntranslatedNewKeys({ baseEn, headEn, headLocales }) {
const base = flattenLeaves(baseEn);
const head = flattenLeaves(headEn);
const newKeys = Object.keys(head).filter(
(k) => !(k in base) && typeof head[k] === "string" && head[k].trim() !== ""
);
if (!newKeys.length) return [];
const gaps = [];
for (const [locale, catalog] of Object.entries(headLocales)) {
const flat = flattenLeaves(catalog);
for (const key of newKeys) {
const value = flat[key];
const satisfied =
typeof value === "string" && (value.trim() !== "" || value.startsWith(PLACEHOLDER_PREFIX));
if (!satisfied) gaps.push({ key, locale });
}
}
gaps.sort((a, b) => a.key.localeCompare(b.key) || a.locale.localeCompare(b.locale));
return gaps;
}
function git(args) {
return execFileSync("git", args, {
cwd: ROOT,
encoding: "utf8",
stdio: ["ignore", "pipe", "pipe"],
});
}
function readPackageVersion() {
try {
return JSON.parse(fs.readFileSync(path.join(ROOT, "package.json"), "utf8")).version;
} catch {
return null;
}
}
function defaultBaseRef() {
const v = readPackageVersion();
return v && /^\d+\.\d+\.\d+$/.test(v) ? `origin/release/v${v}` : null;
}
function resolveDiffBase(baseRef) {
try {
return git(["merge-base", "HEAD", baseRef]).trim();
} catch {
return baseRef;
}
}
function readCatalogAtRef(ref, relPath) {
try {
return JSON.parse(git(["show", `${ref}:${relPath}`]));
} catch {
return null;
}
}
function main() {
const argv = process.argv.slice(2);
const opts = { json: argv.includes("--json"), warn: argv.includes("--warn") };
const baseRef = process.env.BASE_REF || defaultBaseRef();
const skip = (reason) => {
if (opts.json) process.stdout.write(JSON.stringify({ ok: true, skipped: true, reason }) + "\n");
else console.log(`[i18n-new-keys] SKIP reason=${reason}`);
process.exit(0);
};
if (!baseRef) skip("base-unresolved");
const base = resolveDiffBase(baseRef);
const baseEn = readCatalogAtRef(base, `${MESSAGES_REL}/en.json`);
if (!baseEn) skip("base-catalog-unreadable");
const dir = path.join(ROOT, MESSAGES_REL);
const headEn = JSON.parse(fs.readFileSync(path.join(dir, "en.json"), "utf8"));
const headLocales = {};
for (const file of fs.readdirSync(dir)) {
if (!file.endsWith(".json") || file === "en.json") continue;
try {
headLocales[file.replace(/\.json$/, "")] = JSON.parse(
fs.readFileSync(path.join(dir, file), "utf8")
);
} catch {
/* a malformed catalog is another gate's problem */
}
}
const gaps = findUntranslatedNewKeys({ baseEn, headEn, headLocales });
if (opts.json) {
console.log(JSON.stringify({ ok: gaps.length === 0, gaps }, null, 2));
}
if (!gaps.length) {
console.log(
`[i18n-new-keys] PASS — every key new in English reached all ${Object.keys(headLocales).length} locale(s).`
);
return;
}
const byKey = new Map();
for (const g of gaps) {
if (!byKey.has(g.key)) byKey.set(g.key, []);
byKey.get(g.key).push(g.locale);
}
const label = opts.warn ? "WARN" : "FAIL";
console.error(
`\n[i18n-new-keys] ${label}${byKey.size} new English key(s) missing from some locales:`
);
for (const [key, locales] of byKey) {
console.error(`${key} — missing in ${locales.length}: ${locales.join(", ")}`);
}
console.error(
"\n Translate them, or set `__MISSING__:<english>` to defer (the runtime then falls back\n" +
" to English). `vi` bans placeholders — it needs a real translation."
);
if (!opts.warn) process.exit(1);
}
if (import.meta.url === `file://${process.argv[1]}`) main();

View File

@@ -0,0 +1,67 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import fs from "node:fs";
import path from "node:path";
import { parseExclusions, findViolations } from "../../scripts/check/check-vitest-exclusions.mjs";
const ROOT = path.resolve(import.meta.dirname, "..", "..");
/** Every excluded file in the inventory is a real path — a stale entry excludes nothing. */
test("the checked-in inventory only lists files that exist", () => {
const inv = JSON.parse(
fs.readFileSync(path.join(ROOT, "config/quality/vitest-exclusions.json"), "utf8")
);
assert.ok(inv.excluded.length > 0, "inventory is not empty");
for (const entry of inv.excluded) {
assert.ok(fs.existsSync(path.join(ROOT, entry.file)), `${entry.file} exists`);
assert.match(entry.issue, /^#\d+$/, `${entry.file} names a tracking issue`);
}
});
test("the live config satisfies the gate", () => {
const entries = parseExclusions(fs.readFileSync(path.join(ROOT, "vitest.config.ts"), "utf8"));
const inv = JSON.parse(
fs.readFileSync(path.join(ROOT, "config/quality/vitest-exclusions.json"), "utf8")
);
const v = findViolations(
entries,
(p) => fs.existsSync(path.join(ROOT, p)),
inv.excluded.map((e: { file: string }) => e.file)
);
assert.deepEqual(v, { unreferenced: [], untracked: [], orphaned: [] });
});
test("parseExclusions keeps each entry's trailing comment", () => {
const src = ` test: {\n exclude: [\n "node_modules/**",\n "tests/a.test.ts", // #123 — reason\n ]\n }`;
assert.deepEqual(parseExclusions(src), [
{ pattern: "node_modules/**", comment: "," },
{ pattern: "tests/a.test.ts", comment: ", // #123 — reason" },
]);
});
test("an exclusion with no issue reference fails the gate", () => {
const entries = [{ pattern: "tests/a.test.ts", comment: ", // just because" }];
const v = findViolations(entries, () => true, ["tests/a.test.ts"]);
assert.deepEqual(v.unreferenced, ["tests/a.test.ts"]);
});
test("an exclusion absent from the inventory fails the gate", () => {
const entries = [{ pattern: "tests/a.test.ts", comment: ", // #123" }];
const v = findViolations(entries, () => true, []);
assert.deepEqual(v.untracked, ["tests/a.test.ts"]);
});
test("an inventory entry that is no longer excluded fails the gate", () => {
const v = findViolations([], () => true, ["tests/revived.test.ts"]);
assert.deepEqual(v.orphaned, ["tests/revived.test.ts"]);
});
test("tooling exclusions and globs are exempt, and a stale path is ignored", () => {
const entries = [
{ pattern: "node_modules/**", comment: "," },
{ pattern: "tests/unit/**/*.test.tsx", comment: "," },
{ pattern: "tests/deleted.test.ts", comment: "," },
];
const v = findViolations(entries, (p) => p !== "tests/deleted.test.ts", []);
assert.deepEqual(v, { unreferenced: [], untracked: [], orphaned: [] });
});

View File

@@ -0,0 +1,78 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { findUntranslatedNewKeys } from "../../scripts/i18n/check-new-key-coverage.mjs";
const en = (extra: Record<string, string> = {}) => ({ ui: { existing: "Existing", ...extra } });
test("a key that already existed is never flagged, however bad its translations", () => {
const gaps = findUntranslatedNewKeys({
baseEn: en(),
headEn: en(),
headLocales: { pt: { ui: {} } },
});
assert.deepEqual(gaps, [], "pre-existing debt stays frozen — this gate judges only new keys");
});
test("a new English key missing from a locale is flagged", () => {
const gaps = findUntranslatedNewKeys({
baseEn: en(),
headEn: en({ fresh: "Fresh" }),
headLocales: { pt: { ui: { existing: "Existente" } }, de: { ui: { existing: "Vorhanden" } } },
});
assert.deepEqual(gaps, [
{ key: "ui.fresh", locale: "de" },
{ key: "ui.fresh", locale: "pt" },
]);
});
test("a new key translated everywhere passes", () => {
const gaps = findUntranslatedNewKeys({
baseEn: en(),
headEn: en({ fresh: "Fresh" }),
headLocales: { pt: { ui: { existing: "Existente", fresh: "Novo" } } },
});
assert.deepEqual(gaps, []);
});
test("a __MISSING__ placeholder satisfies the gate — it is the documented deferral", () => {
const gaps = findUntranslatedNewKeys({
baseEn: en(),
headEn: en({ fresh: "Fresh" }),
headLocales: { pt: { ui: { existing: "Existente", fresh: "__MISSING__:Fresh" } } },
});
assert.deepEqual(gaps, []);
});
test("an empty string does NOT satisfy the gate", () => {
const gaps = findUntranslatedNewKeys({
baseEn: en(),
headEn: en({ fresh: "Fresh" }),
headLocales: { pt: { ui: { existing: "Existente", fresh: " " } } },
});
assert.deepEqual(gaps, [{ key: "ui.fresh", locale: "pt" }]);
});
/**
* The incident this gate encodes: Phase 3 added keys against 42 locales; the EU batch then
* took the repo to 51, and the nine newcomers never received them.
*/
test("a locale added AFTER the key still has to carry it", () => {
const gaps = findUntranslatedNewKeys({
baseEn: en(),
headEn: en({ compareMode: "Compare runs" }),
headLocales: {
pt: { ui: { existing: "Existente", compareMode: "Comparar execuções" } },
el: { ui: { existing: "Υπάρχον" } },
},
});
assert.deepEqual(gaps, [{ key: "ui.compareMode", locale: "el" }]);
});
test("a new key whose English value is empty is not enforced", () => {
const gaps = findUntranslatedNewKeys({
baseEn: en(),
headEn: en({ blank: "" }),
headLocales: { pt: { ui: { existing: "Existente" } } },
});
assert.deepEqual(gaps, []);
});

View File

@@ -38,68 +38,17 @@ export default defineConfig({
"tests/e2e/ecosystem.test.ts",
"tests/e2e/protocol-clients.test.ts",
// ── Pre-existing failures tracked by #8618 ───────────────────────────────
"open-sse/services/autoCombo/__tests__/providerDiversity.test.ts", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/compareView.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/model-select-modal-keep-open.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/model-select-field-6540.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/issue-7845-log-detail-structured-error.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/allocation-table.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/namedCombos-active-badge.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/providerIconKimiLogomark.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/ClaudeClassifierCompatToggle.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/system-storage-manual-vacuum.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/burn-rate-chart.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/model-select-modal-zero-config.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/model-select-modal-hidden-models-7156.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/toonEncoderTable.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/playView.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/studioTabs.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/studio-pages.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/livePage.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/diffPane.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/CliCodePage.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/engineConfigPage.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"open-sse/services/autoCombo/__tests__/autoCombo.test.ts", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/agent-card-risk-modal.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/request-logger-autorefresh-visibility-3972.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/search-tools-compare-tab.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/noauth-account-card.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/playground-build-tab.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/playground-studio.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/playground-compare-tab.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"src/app/(dashboard)/dashboard/webhooks/__tests__/webhook-wizard.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/search-tools-scrape-result.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/CliToolCard.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/comboLiveStudio.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"src/app/(dashboard)/dashboard/cache/__tests__/CachePage.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"src/lib/memory/__tests__/retrieval.test.ts", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/model-select-modal-select-all.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/model-select-modal-deselect.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/engine-pages.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/playground-config-pane.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/logs-page-detail-modal-reopen-on-close.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/agent-card.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/agent-bridge-page.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/model-select-modal-connection-filter.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/playground-structured-output-editor.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/playground-tools-builder.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/playground-improve-prompt-button.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"src/app/(dashboard)/dashboard/endpoint/__tests__/ApiEndpointsTab.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/compression-combos-routing-mode-6760.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/use-local-storage-pool-migration.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/waterfallInspector.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/playground-compare-column.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/playground-chat-tab.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"src/app/(dashboard)/dashboard/providers/[id]/__tests__/ProviderDetailPageClient.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"src/lib/skills/__tests__/integration.test.ts", // #8618 — pre-existing failure; remove this exclusion when fixed
"src/app/(dashboard)/dashboard/cache/__tests__/CacheTrends.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"src/app/(dashboard)/dashboard/cache/__tests__/IdempotencyLayer.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"src/app/(dashboard)/dashboard/cache/__tests__/CachePerformance.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"src/app/(dashboard)/dashboard/cache/__tests__/MemoryCards.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"src/app/(dashboard)/dashboard/discovery/__tests__/DiscoveryPageClient.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"open-sse/services/autoCombo/__tests__/chaosVirtualCombo.test.ts", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/combos-page-smoke.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/evals-tab-smoke.test.tsx", // #8618 — pre-existing failure; remove this exclusion when fixed
"tests/unit/ui/request-logger-autorefresh-visibility-3972.test.tsx", // #13204 — falha real; remover esta exclusão quando consertado
"src/app/(dashboard)/dashboard/webhooks/__tests__/webhook-wizard.test.tsx", // #13204 — falha real; remover esta exclusão quando consertado
"tests/unit/ui/logs-page-detail-modal-reopen-on-close.test.tsx", // #13204 — falha real; remover esta exclusão quando consertado
"tests/unit/ui/agent-card.test.tsx", // #13204 — falha real; remover esta exclusão quando consertado
"src/app/(dashboard)/dashboard/endpoint/__tests__/ApiEndpointsTab.test.tsx", // #13204 — falha real; remover esta exclusão quando consertado
"src/app/(dashboard)/dashboard/cache/__tests__/CacheTrends.test.tsx", // #13204 — falha real; remover esta exclusão quando consertado
"src/app/(dashboard)/dashboard/cache/__tests__/IdempotencyLayer.test.tsx", // #13204 — falha real; remover esta exclusão quando consertado
"src/app/(dashboard)/dashboard/cache/__tests__/CachePerformance.test.tsx", // #13204 — falha real; remover esta exclusão quando consertado
"src/app/(dashboard)/dashboard/discovery/__tests__/DiscoveryPageClient.test.tsx", // #13204 — falha real; remover esta exclusão quando consertado
"tests/unit/ui/combos-page-smoke.test.tsx", // #13204 — falha real; remover esta exclusão quando consertado
"tests/unit/ui/evals-tab-smoke.test.tsx", // #13204 — falha real; remover esta exclusão quando consertado
],
coverage: {