mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-07-26 09:52:11 +03:00
feat(ci): block stale UI translations when an English value is rewritten
Closes the gap that let #8463 ship. `oauthModal.googleOAuthWarning`'s English value was rewritten when the Antigravity login helper landed (#5203); 39 of 43 locales kept a translation of the PREVIOUS English, which told operators to "copy the full URL and paste it below" — a flow that cannot complete for that provider family. Non-English users read confident, wrong instructions for months and no gate noticed. None of the three existing gates can see this class: - `sync-ui-keys.mjs` only backfills keys that are ABSENT, never ones that are STALE; - `check-ui-keys-coverage.mjs` counts key PRESENCE, so a stale translation scores as fully covered (all 43 locales sat at 99.6% throughout); - `check-translation-drift.mjs` tracks the `docs/i18n/<locale>/**.md` documentation mirrors — it never reads `src/i18n/messages/*.json` at all. (Its `.i18n-state.json` is also absent, so it self-skips, but bootstrapping it would not have helped: wrong surface.) New gate `scripts/i18n/check-ui-value-drift.mjs` is DIFF-AWARE rather than baseline-backed: it compares `en.json` at the merge base against the working tree, and for every key whose English value changed, reports any locale still holding an untouched translation. That choice deliberately freezes pre-existing debt — a diff cannot reveal which old English a long-standing translation came from, so the gate judges only what the current change touches, and unrelated PRs never pay for historical drift. The alternative, a per-key hash baseline over 11207 keys, would have cost a ~600 KB generated file (3x the largest existing baseline) churning on every i18n PR. Two ways to satisfy it: refresh the translations, or set them to `__MISSING__:<new english>` so the runtime serves the corrected English (#7258) while the key queues for translation. When the string's MEANING changes, renaming the key is better still — a new key cannot inherit a stale translation, which is what #8463 did. Wired blocking into the `i18n-ui-coverage` job (the `i18n` job is `continue-on-error: true`, so a gate there could not block anything). That job gains `fetch-depth: 0` because the gate needs the base ref; without it the gate self-skips with `base-unresolved`, mirroring `check-openapi-breaking`. `BASE_REF` is passed via `env:` and reaches git only through `execFileSync` argv — never a shell string. Verified against the real defect: rewriting an English value with translations left behind reports exactly 39 stale locales and exits 1; `--warn` exits 0; an unresolvable base exits 0 with `SKIP reason=base-unresolved`.
This commit is contained in:
ikelvingo
11
.github/workflows/ci.yml
vendored
11
.github/workflows/ci.yml
vendored
@@ -449,15 +449,26 @@ jobs:
|
||||
# UI keys move with dashboard code OR message catalogs.
|
||||
if: ${{ github.event_name != 'pull_request' || (github.event.pull_request.draft == false && (needs.changes.outputs.i18n == 'true' || needs.changes.outputs.code == 'true')) }}
|
||||
steps:
|
||||
# fetch-depth: 0 — the value-drift gate diffs en.json against the merge base to
|
||||
# find rewritten English strings. On a shallow clone the base ref is missing and
|
||||
# the gate self-skips (base-unresolved), so it would never actually run.
|
||||
- uses: actions/checkout@v7
|
||||
with:
|
||||
persist-credentials: false
|
||||
fetch-depth: 0
|
||||
- uses: actions/setup-node@v7
|
||||
with:
|
||||
node-version: ${{ env.CI_NODE_VERSION }}
|
||||
cache: npm
|
||||
- uses: ./.github/actions/npm-ci-retry
|
||||
- run: node scripts/i18n/check-ui-keys-coverage.mjs --threshold=65
|
||||
# #8463: a rewritten English value used to leave its 39 translations behind
|
||||
# silently (googleOAuthWarning shipped wrong copy in 39 locales for months).
|
||||
# Key parity above cannot see it — a stale translation counts as covered.
|
||||
- name: i18n UI value drift (stale translations)
|
||||
env:
|
||||
BASE_REF: ${{ github.base_ref && format('origin/{0}', github.base_ref) || '' }}
|
||||
run: node scripts/i18n/check-ui-value-drift.mjs
|
||||
|
||||
# #8038: cheap single-locale glossary/protected-terms consistency gate —
|
||||
# complements i18n-ui-coverage (key parity) and the ICU `i18n` job below
|
||||
|
||||
@@ -22,30 +22,30 @@ The CI source of truth is `.github/workflows/ci.yml`.
|
||||
|
||||
Runs on every PR to `main`. Blocks merge on failure.
|
||||
|
||||
| Script (`npm run ...`) | Validates | Blocking |
|
||||
| ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------- |
|
||||
| `check:node-runtime` | Node.js version is within the supported range | Yes |
|
||||
| `check:cycles` | Circular imports — all `src/` + `open-sse/` modules | Yes |
|
||||
| `check:route-validation:t06` | Zod schemas present on all routes (Tier 6 policy) | Yes |
|
||||
| `check:any-budget:t11` | `@ts-expect-error // any` count does not exceed budget (Tier 11 catraca) | Yes |
|
||||
| `check:provider-consistency` | Every provider in `providers.ts` has a matching entry in `providerRegistry.ts` (and vice-versa, within the allowlist) | Yes |
|
||||
| `check:fetch-targets` | Every `fetch("/api/...")` in client-side `src/` resolves to a real `route.ts` | Yes |
|
||||
| `check:deps` | All `npm install`-able deps across every `package.json` in the repo are in `dependency-allowlist.json`; new unpinned or slopsquatted packages flagged | Yes |
|
||||
| `audit:deps` | `npm audit` (root + electron) — no high/critical advisories (overlaps osv `check:vuln-ratchet`; see Rationalization Backlog) | Yes |
|
||||
| `check:lockfile` | `package-lock.json` integrity — https registry, integrity hashes, no host overrides | Yes |
|
||||
| `check:licenses` | SPDX license allowlist for production dependencies | Yes |
|
||||
| `check:tracked-artifacts` | No build artifacts / committed `node_modules` symlinks (also runs in husky pre-commit; pre-push is intentionally light — #6716) | Yes |
|
||||
| `check:file-size` | No source file exceeds the per-extension cap (ratchet: frozen large files in `frozen` list) | Yes |
|
||||
| `check:error-helper` | Error responses in executors/handlers use `buildErrorBody()` / `sanitizeErrorMessage()` (Hard Rule #12) | Yes |
|
||||
| `check:migration-numbering` | Migration SQL files are sequentially numbered, no gaps or duplicates | Yes |
|
||||
| `check:public-creds` | No literal OAuth `client_id`/`client_secret` or Firebase Web keys outside `publicCreds.ts` (Hard Rule #11) | Yes |
|
||||
| `check:db-rules` | No raw SQL outside `src/lib/db/` modules; no barrel-imports from `localDb.ts` (Hard Rules #2/#5) | Yes |
|
||||
| `check:known-symbols` | Provider executors, routing strategies, and translators registered in their dispatch tables match the files on disk — no orphaned or undeclared symbols | Yes |
|
||||
| `check:route-guard-membership` | Every route that spawns a child process is classified by `isLocalOnlyPath()` (Hard Rules #15/#17) | Yes |
|
||||
| `check:test-discovery` | Every `*.test.ts` / `*.spec.ts` file in the repo is collected by at least one test runner (ratchet: orphan list in `test-discovery-baseline.json` can only shrink) | Yes |
|
||||
| `check:docs-sync` | CHANGELOG version, OpenAPI version, and `llm.txt` are in sync | Yes |
|
||||
| `typecheck:core` | TypeScript compilation without errors (advisory warnings only) | Yes |
|
||||
| `typecheck:noimplicit:core` | Strict `noImplicitAny` — forward-looking; many pre-existing call sites still need annotations | **Advisory** (`continue-on-error: true`) |
|
||||
| Script (`npm run ...`) | Validates | Blocking |
|
||||
| ------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------- |
|
||||
| `check:node-runtime` | Node.js version is within the supported range | Yes |
|
||||
| `check:cycles` | Circular imports — all `src/` + `open-sse/` modules | Yes |
|
||||
| `check:route-validation:t06` | Zod schemas present on all routes (Tier 6 policy) | Yes |
|
||||
| `check:any-budget:t11` | `@ts-expect-error // any` count does not exceed budget (Tier 11 catraca) | Yes |
|
||||
| `check:provider-consistency` | Every provider in `providers.ts` has a matching entry in `providerRegistry.ts` (and vice-versa, within the allowlist) | Yes |
|
||||
| `check:fetch-targets` | Every `fetch("/api/...")` in client-side `src/` resolves to a real `route.ts` | Yes |
|
||||
| `check:deps` | All `npm install`-able deps across every `package.json` in the repo are in `dependency-allowlist.json`; new unpinned or slopsquatted packages flagged | Yes |
|
||||
| `audit:deps` | `npm audit` (root + electron) — no high/critical advisories (overlaps osv `check:vuln-ratchet`; see Rationalization Backlog) | Yes |
|
||||
| `check:lockfile` | `package-lock.json` integrity — https registry, integrity hashes, no host overrides | Yes |
|
||||
| `check:licenses` | SPDX license allowlist for production dependencies | Yes |
|
||||
| `check:tracked-artifacts` | No build artifacts / committed `node_modules` symlinks (also runs in husky pre-commit; pre-push is intentionally light — #6716) | Yes |
|
||||
| `check:file-size` | No source file exceeds the per-extension cap (ratchet: frozen large files in `frozen` list) | Yes |
|
||||
| `check:error-helper` | Error responses in executors/handlers use `buildErrorBody()` / `sanitizeErrorMessage()` (Hard Rule #12) | Yes |
|
||||
| `check:migration-numbering` | Migration SQL files are sequentially numbered, no gaps or duplicates | Yes |
|
||||
| `check:public-creds` | No literal OAuth `client_id`/`client_secret` or Firebase Web keys outside `publicCreds.ts` (Hard Rule #11) | Yes |
|
||||
| `check:db-rules` | No raw SQL outside `src/lib/db/` modules; no barrel-imports from `localDb.ts` (Hard Rules #2/#5) | Yes |
|
||||
| `check:known-symbols` | Provider executors, routing strategies, and translators registered in their dispatch tables match the files on disk — no orphaned or undeclared symbols | Yes |
|
||||
| `check:route-guard-membership` | Every route that spawns a child process is classified by `isLocalOnlyPath()` (Hard Rules #15/#17) | Yes |
|
||||
| `check:test-discovery` | Every `*.test.ts` / `*.spec.ts` file in the repo is collected by at least one test runner (ratchet: orphan list in `test-discovery-baseline.json` can only shrink) | Yes |
|
||||
| `check:docs-sync` | CHANGELOG version, OpenAPI version, and `llm.txt` are in sync | Yes |
|
||||
| `typecheck:core` | TypeScript compilation without errors (advisory warnings only) | Yes |
|
||||
| `typecheck:noimplicit:core` | Strict `noImplicitAny` — forward-looking; many pre-existing call sites still need annotations | **Advisory** (`continue-on-error: true`) |
|
||||
| `check:dashboard-typecheck` | `tsc` scoped to `src/app/(dashboard)/**` (#7033) — `typecheck:core`'s curated 27-file allowlist does not include any dashboard TSX, and `next build` never type-checks it either (`next.config.mjs` sets `ignoreBuildErrors: true`), so orphaned-identifier regressions there (#6625/#6909) were invisible to CI. Diffs against a frozen per-file/per-TS-code count baseline (`config/quality/dashboard-typecheck-baseline.json`, same stale-enforcement pattern as `check:known-symbols`) — only NEW errors beyond the baselined count fail the gate; ratchet down with `--update` when a pre-existing error is fixed. | Yes |
|
||||
|
||||
### Job: `quality-gate`
|
||||
@@ -100,9 +100,53 @@ Runs on every PR to `main`. Blocks merge on failure.
|
||||
|
||||
### Job: `i18n-ui-coverage`
|
||||
|
||||
| Script | Validates | Blocking |
|
||||
| --------------------------------- | ----------------------------- | -------- |
|
||||
| `check-ui-keys-coverage` (inline) | UI i18n key coverage is ≥ 65% | Yes |
|
||||
| Script | Validates | Blocking |
|
||||
| --------------------------------- | ---------------------------------------------------------------- | -------- |
|
||||
| `check-ui-keys-coverage` (inline) | UI i18n key coverage is ≥ 65% | Yes |
|
||||
| `check-ui-value-drift` (inline) | A rewritten English **value** leaves no stale translation behind | Yes |
|
||||
|
||||
Needs `fetch-depth: 0` — the value-drift gate diffs `en.json` against the merge base.
|
||||
|
||||
#### `check-ui-value-drift` — stale-translation gate
|
||||
|
||||
Catches the one i18n regression the other gates structurally cannot see: an English value
|
||||
is rewritten and the translations derived from the _previous_ English stay behind, so
|
||||
non-English users keep reading confidently-worded, now-wrong copy.
|
||||
|
||||
This shipped for real. `oauthModal.googleOAuthWarning` was rewritten when the Antigravity
|
||||
login helper landed (#5203); **39 of 43 locales** kept text telling operators to "copy the
|
||||
full URL and paste it below" — a flow that cannot complete for that provider. It went
|
||||
unnoticed until #8463 because:
|
||||
|
||||
- `sync-ui-keys` only backfills keys that are **absent**, never ones that are **stale**;
|
||||
- `check-ui-keys-coverage` counts key _presence_, so a stale translation scores as covered;
|
||||
- `check-translation-drift` tracks the `docs/i18n/<locale>/**.md` documentation mirrors —
|
||||
it never reads `src/i18n/messages/*.json`.
|
||||
|
||||
**Diff-aware, not baseline-backed.** It compares `en.json` at the merge base against the
|
||||
working tree; for every key whose English value changed, any locale still holding an
|
||||
untouched translation is stale. This deliberately **freezes pre-existing debt** — a diff
|
||||
cannot reveal which old English a long-standing translation came from, so the gate judges
|
||||
only what the current change touches. The alternative (a per-key hash baseline) would cost
|
||||
a ~600 KB generated file, 3× the largest existing baseline, churning on every i18n PR.
|
||||
|
||||
Two ways to satisfy it:
|
||||
|
||||
1. update the affected translations, or
|
||||
2. set them to `__MISSING__:<new english>` — the runtime then serves the corrected English
|
||||
(`src/i18n/request.ts::deepMergeFallback`, #7258) and the key queues for translation.
|
||||
|
||||
If the string's **meaning** changed, prefer **renaming the key**: a new key cannot inherit
|
||||
a stale translation. That is the pattern #8463 used.
|
||||
|
||||
```bash
|
||||
npm run i18n:check-value-drift # strict (what CI runs)
|
||||
npm run i18n:check-value-drift:warn # report only
|
||||
BASE_REF=origin/release/vX.Y.Z npm run i18n:check-value-drift
|
||||
```
|
||||
|
||||
Exits 0 with `SKIP reason=base-unresolved` when the base catalog cannot be read (shallow
|
||||
clone without the base ref), mirroring `check-openapi-breaking`.
|
||||
|
||||
### Job: `i18n`
|
||||
|
||||
@@ -179,11 +223,11 @@ pending implementation).
|
||||
Retry is per-runner, never a global blanket — a blanket retry converts real regressions
|
||||
into invisible flakes:
|
||||
|
||||
| Runner | Policy | Why |
|
||||
| --- | --- | --- |
|
||||
| Playwright (e2e) | `retries: 1` in CI only, with `trace: on-first-retry` | Browser/network timing is genuinely nondeterministic; one retry with a trace turns a flake into a diagnosable artifact |
|
||||
| Vitest | NO global retry. A proven-flaky test gets an explicit per-test retry (visible in the diff, reviewed in PR) | Keeps the quarantine list in the repo, never opaque |
|
||||
| node:test (unit) | NO retry, ever | A flaky unit test is a bug in the test — fix it, don't re-roll it |
|
||||
| Runner | Policy | Why |
|
||||
| ---------------- | ---------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
|
||||
| Playwright (e2e) | `retries: 1` in CI only, with `trace: on-first-retry` | Browser/network timing is genuinely nondeterministic; one retry with a trace turns a flake into a diagnosable artifact |
|
||||
| Vitest | NO global retry. A proven-flaky test gets an explicit per-test retry (visible in the diff, reviewed in PR) | Keeps the quarantine list in the repo, never opaque |
|
||||
| node:test (unit) | NO retry, ever | A flaky unit test is a bug in the test — fix it, don't re-roll it |
|
||||
|
||||
Target SLOs once flake telemetry lands (WS5.2/5.3): <1% flake rate per test
|
||||
("fix now" threshold), ≥95% pass rate per pipeline. Industry reference values —
|
||||
|
||||
@@ -136,6 +136,8 @@
|
||||
"i18n:sync-ui": "node scripts/i18n/sync-ui-keys.mjs",
|
||||
"i18n:sync-ui:dry": "node scripts/i18n/sync-ui-keys.mjs --dry-run",
|
||||
"i18n:check-ui-coverage": "node scripts/i18n/check-ui-keys-coverage.mjs",
|
||||
"i18n:check-value-drift": "node scripts/i18n/check-ui-value-drift.mjs",
|
||||
"i18n:check-value-drift:warn": "node scripts/i18n/check-ui-value-drift.mjs --warn",
|
||||
"i18n:check-glossary": "node scripts/i18n/check-glossary-consistency.mjs",
|
||||
"check:node-runtime": "node --import tsx scripts/check/check-supported-node-runtime.ts",
|
||||
"check:pack-artifact": "node --import tsx scripts/build/validate-pack-artifact.ts",
|
||||
|
||||
280
scripts/i18n/check-ui-value-drift.mjs
Normal file
280
scripts/i18n/check-ui-value-drift.mjs
Normal file
@@ -0,0 +1,280 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* OmniRoute — UI i18n VALUE-drift gate (CI gate, blocking).
|
||||
*
|
||||
* Catches the one i18n regression no existing gate can see: an English **value** is
|
||||
* rewritten and the translations derived from the previous English are left behind, so
|
||||
* every non-English user keeps reading confidently-worded, now-wrong copy.
|
||||
*
|
||||
* Real incident (fixed in #8463): `oauthModal.googleOAuthWarning` was rewritten when the
|
||||
* Antigravity login helper shipped (#5203). 39 of 43 locales kept the old text, which
|
||||
* told operators to "copy the full URL and paste it below" — a flow that cannot complete
|
||||
* for that provider family. Nobody noticed, because:
|
||||
*
|
||||
* - `sync-ui-keys.mjs` only backfills keys that are ABSENT, never ones that are STALE;
|
||||
* - `check-ui-keys-coverage.mjs` counts key presence, so a stale translation scores as
|
||||
* fully covered;
|
||||
* - `check-translation-drift.mjs` tracks the `docs/i18n/<locale>/**.md` documentation
|
||||
* mirrors — it never looks at `src/i18n/messages/*.json` at all.
|
||||
*
|
||||
* How this gate works: it is DIFF-AWARE, not baseline-backed. It compares the English
|
||||
* catalog at the merge base against the working tree; for every key whose English value
|
||||
* changed, any locale still holding an untouched translation of it is stale.
|
||||
*
|
||||
* That deliberately freezes pre-existing debt: a diff cannot reveal which old English a
|
||||
* long-standing translation was derived from, so the gate judges only what the current
|
||||
* change touches. The alternative — a per-key hash baseline — would cost a ~600 KB
|
||||
* generated file (3x the largest existing baseline) that churns on every i18n PR.
|
||||
*
|
||||
* Two ways to satisfy it:
|
||||
* 1. update the affected translations, or
|
||||
* 2. set them to `__MISSING__:<new english>`, which makes the runtime fall back to the
|
||||
* correct English (src/i18n/request.ts::deepMergeFallback, #7258) and queues the key
|
||||
* for the translation pipeline.
|
||||
*
|
||||
* Usage:
|
||||
* node scripts/i18n/check-ui-value-drift.mjs # strict (default), exit 1 on drift
|
||||
* node scripts/i18n/check-ui-value-drift.mjs --warn # report, exit 0
|
||||
* node scripts/i18n/check-ui-value-drift.mjs --json # machine-readable report
|
||||
* BASE_REF=origin/release/vX.Y.Z node scripts/i18n/check-ui-value-drift.mjs
|
||||
*
|
||||
* Graceful SKIP (exit 0) when the base English catalog cannot be resolved — a shallow
|
||||
* clone without the base ref, or a brand-new catalog. Mirrors the base-unresolved SKIP in
|
||||
* `scripts/check/check-openapi-breaking.mjs`.
|
||||
*/
|
||||
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import process from "node:process";
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const SCRIPT_DIR = path.dirname(fileURLToPath(import.meta.url));
|
||||
const ROOT = path.resolve(SCRIPT_DIR, "..", "..");
|
||||
const MESSAGES_REL = "src/i18n/messages";
|
||||
const MESSAGES_DIR = path.join(ROOT, MESSAGES_REL);
|
||||
const PLACEHOLDER_PREFIX = "__MISSING__:";
|
||||
|
||||
/** Flatten a nested message catalog into `{ "a.b.c": value }`. */
|
||||
export function flattenLeaves(node, prefix = "", out = {}) {
|
||||
for (const [key, value] of Object.entries(node ?? {})) {
|
||||
const dotted = prefix ? `${prefix}.${key}` : key;
|
||||
if (value !== null && typeof value === "object" && !Array.isArray(value)) {
|
||||
flattenLeaves(value, dotted, out);
|
||||
} else {
|
||||
out[dotted] = value;
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* Pure core: which (key, locale) pairs carry a translation of a now-changed English value?
|
||||
*
|
||||
* @param {object} args
|
||||
* @param {object} args.baseEn en.json at the base ref
|
||||
* @param {object} args.headEn en.json in the working tree
|
||||
* @param {Record<string, object>} args.baseLocales locale -> catalog at the base ref
|
||||
* @param {Record<string, object>} args.headLocales locale -> catalog in the working tree
|
||||
* @returns {Array<{ key: string, locale: string }>} sorted, stable
|
||||
*/
|
||||
export function findStaleTranslations({ baseEn, headEn, baseLocales, headLocales }) {
|
||||
const baseFlat = flattenLeaves(baseEn);
|
||||
const headFlat = flattenLeaves(headEn);
|
||||
|
||||
// Keys whose English copy was REWRITTEN by this change. A key absent from either side
|
||||
// is an add or a delete: nothing can be stale against text that did not exist, and a
|
||||
// renamed key (delete + add) is exactly the safe fix pattern.
|
||||
const rewritten = Object.keys(headFlat).filter(
|
||||
(key) => key in baseFlat && baseFlat[key] !== headFlat[key]
|
||||
);
|
||||
if (rewritten.length === 0) return [];
|
||||
|
||||
const flatCache = new Map();
|
||||
const flatOf = (bag, locale) => {
|
||||
const cacheKey = `${bag === baseLocales ? "base" : "head"}:${locale}`;
|
||||
if (!flatCache.has(cacheKey)) flatCache.set(cacheKey, flattenLeaves(bag[locale]));
|
||||
return flatCache.get(cacheKey);
|
||||
};
|
||||
|
||||
const stale = [];
|
||||
for (const locale of Object.keys(headLocales)) {
|
||||
if (locale === "en") continue; // the source of truth is never its own target
|
||||
const head = flatOf(headLocales, locale);
|
||||
const base = flatOf(baseLocales, locale);
|
||||
|
||||
for (const key of rewritten) {
|
||||
const headValue = head[key];
|
||||
// Never translated here — that is `check-ui-keys-coverage`'s concern, not ours.
|
||||
if (headValue === undefined) continue;
|
||||
// Explicitly pending: the runtime serves the corrected English instead.
|
||||
if (typeof headValue === "string" && headValue.startsWith(PLACEHOLDER_PREFIX)) continue;
|
||||
// Refreshed alongside the English in this very change.
|
||||
if (base[key] !== headValue) continue;
|
||||
|
||||
stale.push({ key, locale });
|
||||
}
|
||||
}
|
||||
|
||||
return stale.sort((a, b) => a.key.localeCompare(b.key) || a.locale.localeCompare(b.locale));
|
||||
}
|
||||
|
||||
function parseArgs(argv) {
|
||||
const opts = { mode: "strict", json: false };
|
||||
for (const arg of argv.slice(2)) {
|
||||
if (arg === "--warn") opts.mode = "warn";
|
||||
else if (arg === "--strict") opts.mode = "strict";
|
||||
else if (arg === "--json") opts.json = true;
|
||||
else if (arg === "--help" || arg === "-h") {
|
||||
console.log(
|
||||
[
|
||||
"Usage: node scripts/i18n/check-ui-value-drift.mjs [--strict|--warn] [--json]",
|
||||
"",
|
||||
" --strict (default) exit 1 when a rewritten English value leaves a stale translation",
|
||||
" --warn report drift but exit 0",
|
||||
" --json machine-readable report on stdout",
|
||||
"",
|
||||
" BASE_REF ref to diff against (default: derived from package.json version)",
|
||||
].join("\n")
|
||||
);
|
||||
process.exit(0);
|
||||
}
|
||||
}
|
||||
return opts;
|
||||
}
|
||||
|
||||
function readPackageVersion() {
|
||||
try {
|
||||
return JSON.parse(fs.readFileSync(path.join(ROOT, "package.json"), "utf8")).version ?? null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/** `3.8.49` -> `origin/release/v3.8.49`, matching check-openapi-breaking's convention. */
|
||||
function defaultBaseRef() {
|
||||
const version = readPackageVersion();
|
||||
return version && /^\d+\.\d+\.\d+$/.test(version) ? `origin/release/v${version}` : null;
|
||||
}
|
||||
|
||||
function git(args) {
|
||||
return execFileSync("git", args, {
|
||||
cwd: ROOT,
|
||||
encoding: "utf8",
|
||||
stdio: ["ignore", "pipe", "pipe"],
|
||||
});
|
||||
}
|
||||
|
||||
/** Prefer the merge base so a base branch that advanced independently cannot false-positive. */
|
||||
function resolveDiffBase(baseRef) {
|
||||
try {
|
||||
return git(["merge-base", "HEAD", baseRef]).trim();
|
||||
} catch {
|
||||
return baseRef; // shallow clone without common history — the tip is the best available
|
||||
}
|
||||
}
|
||||
|
||||
function readCatalogAtRef(ref, relPath) {
|
||||
try {
|
||||
return JSON.parse(git(["show", `${ref}:${relPath}`]));
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function readCatalogFromDisk(absPath) {
|
||||
try {
|
||||
return JSON.parse(fs.readFileSync(absPath, "utf8"));
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function main() {
|
||||
const opts = parseArgs(process.argv);
|
||||
const baseRef = process.env.BASE_REF || defaultBaseRef();
|
||||
|
||||
const emitSkip = (reason) => {
|
||||
if (opts.json) {
|
||||
process.stdout.write(JSON.stringify({ ok: true, skipped: true, reason }) + "\n");
|
||||
} else {
|
||||
console.log(`[i18n-value-drift] SKIP reason=${reason}`);
|
||||
}
|
||||
process.exit(0);
|
||||
};
|
||||
|
||||
if (!baseRef) emitSkip("base-unresolved");
|
||||
|
||||
const diffBase = resolveDiffBase(baseRef);
|
||||
const baseEn = readCatalogAtRef(diffBase, `${MESSAGES_REL}/en.json`);
|
||||
if (!baseEn) emitSkip("base-unresolved");
|
||||
|
||||
const headEn = readCatalogFromDisk(path.join(MESSAGES_DIR, "en.json"));
|
||||
if (!headEn) emitSkip("head-en-unreadable");
|
||||
|
||||
const locales = fs
|
||||
.readdirSync(MESSAGES_DIR)
|
||||
.filter((f) => f.endsWith(".json"))
|
||||
.map((f) => f.slice(0, -5));
|
||||
|
||||
const headLocales = {};
|
||||
const baseLocales = {};
|
||||
for (const locale of locales) {
|
||||
headLocales[locale] = readCatalogFromDisk(path.join(MESSAGES_DIR, `${locale}.json`)) ?? {};
|
||||
baseLocales[locale] = readCatalogAtRef(diffBase, `${MESSAGES_REL}/${locale}.json`) ?? {};
|
||||
}
|
||||
|
||||
const stale = findStaleTranslations({ baseEn, headEn, baseLocales, headLocales });
|
||||
|
||||
if (opts.json) {
|
||||
process.stdout.write(
|
||||
JSON.stringify({ ok: stale.length === 0, baseRef, diffBase, stale }, null, 2) + "\n"
|
||||
);
|
||||
process.exit(stale.length === 0 || opts.mode === "warn" ? 0 : 1);
|
||||
}
|
||||
|
||||
if (stale.length === 0) {
|
||||
console.log(
|
||||
`[i18n-value-drift] PASS — no rewritten English value left a stale translation (base ${baseRef}).`
|
||||
);
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
// Group by key so the report reads as "this string changed; these locales lag".
|
||||
const byKey = new Map();
|
||||
for (const { key, locale } of stale) {
|
||||
if (!byKey.has(key)) byKey.set(key, []);
|
||||
byKey.get(key).push(locale);
|
||||
}
|
||||
|
||||
console.log(
|
||||
`[i18n-value-drift] ${byKey.size} English value(s) rewritten with ${stale.length} stale translation(s):`
|
||||
);
|
||||
for (const [key, affected] of byKey) {
|
||||
console.log(` - ${key}`);
|
||||
console.log(` stale in ${affected.length} locale(s): ${affected.join(", ")}`);
|
||||
}
|
||||
console.log("");
|
||||
console.log(" Fix either way:");
|
||||
console.log(" 1. update those translations to match the new English, or");
|
||||
console.log(` 2. set them to "${PLACEHOLDER_PREFIX}<new english>" so the runtime serves`);
|
||||
console.log(" the corrected English until the translation pipeline catches up.");
|
||||
console.log("");
|
||||
console.log(" If the string's MEANING changed, prefer renaming the key instead — a new key");
|
||||
console.log(" cannot inherit a stale translation (see #8463).");
|
||||
|
||||
if (opts.mode === "warn") {
|
||||
console.log("[i18n-value-drift] WARN — drift detected (warn mode, exiting 0).");
|
||||
process.exit(0);
|
||||
}
|
||||
console.log("[i18n-value-drift] FAIL — stale translations detected.");
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// Only run the CLI when invoked directly, so the pure helpers stay importable in tests.
|
||||
if (
|
||||
process.argv[1] &&
|
||||
path.resolve(process.argv[1]) === path.resolve(fileURLToPath(import.meta.url))
|
||||
) {
|
||||
main();
|
||||
}
|
||||
160
tests/unit/i18n-ui-value-drift.test.ts
Normal file
160
tests/unit/i18n-ui-value-drift.test.ts
Normal file
@@ -0,0 +1,160 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
// @ts-expect-error — plain .mjs gate script, no type declarations by design.
|
||||
import { findStaleTranslations, flattenLeaves } from "../../scripts/i18n/check-ui-value-drift.mjs";
|
||||
|
||||
// Why this gate exists (the #8463 defect):
|
||||
//
|
||||
// `googleOAuthWarning`'s English value was rewritten when the Antigravity login helper
|
||||
// shipped (#5203). Nothing noticed that 39 of 43 locales still carried a translation of
|
||||
// the PREVIOUS English — which told operators to "copy the full URL and paste it below",
|
||||
// a flow that cannot complete for that provider. Users saw confident, wrong instructions
|
||||
// in their own language for months.
|
||||
//
|
||||
// Neither existing gate can see this:
|
||||
// - `sync-ui-keys.mjs` only fills keys that are ABSENT, never ones that are STALE;
|
||||
// - `check-ui-keys-coverage.mjs` counts key presence, so a stale translation scores
|
||||
// as 100% covered;
|
||||
// - `check-translation-drift.mjs` tracks `docs/i18n/<locale>/**.md` (documentation
|
||||
// mirrors), not the UI message catalogs at all.
|
||||
//
|
||||
// So: when a PR edits an English VALUE, every locale still holding a translation derived
|
||||
// from the old English is stale and must either be refreshed or flagged
|
||||
// `__MISSING__:` (which makes the runtime fall back to the correct English, #7258).
|
||||
|
||||
test("flattenLeaves walks nested catalogs into dotted paths", () => {
|
||||
assert.deepEqual(flattenLeaves({ a: { b: "x" }, c: "y" }), {
|
||||
"a.b": "x",
|
||||
c: "y",
|
||||
});
|
||||
});
|
||||
|
||||
test("an edited EN value with an untouched translation is STALE", () => {
|
||||
const stale = findStaleTranslations({
|
||||
baseEn: { m: { k: "old english" } },
|
||||
headEn: { m: { k: "new english" } },
|
||||
baseLocales: { "pt-BR": { m: { k: "português antigo" } } },
|
||||
headLocales: { "pt-BR": { m: { k: "português antigo" } } },
|
||||
});
|
||||
|
||||
assert.deepEqual(stale, [{ key: "m.k", locale: "pt-BR" }]);
|
||||
});
|
||||
|
||||
test("a translation refreshed in the same diff is NOT stale", () => {
|
||||
const stale = findStaleTranslations({
|
||||
baseEn: { m: { k: "old english" } },
|
||||
headEn: { m: { k: "new english" } },
|
||||
baseLocales: { "pt-BR": { m: { k: "português antigo" } } },
|
||||
headLocales: { "pt-BR": { m: { k: "português novo" } } },
|
||||
});
|
||||
|
||||
assert.deepEqual(stale, []);
|
||||
});
|
||||
|
||||
test("a translation demoted to __MISSING__: is NOT stale (runtime falls back to EN)", () => {
|
||||
const stale = findStaleTranslations({
|
||||
baseEn: { m: { k: "old english" } },
|
||||
headEn: { m: { k: "new english" } },
|
||||
baseLocales: { "pt-BR": { m: { k: "português antigo" } } },
|
||||
headLocales: { "pt-BR": { m: { k: "__MISSING__:new english" } } },
|
||||
});
|
||||
|
||||
assert.deepEqual(stale, []);
|
||||
});
|
||||
|
||||
test("a locale that never had the key is NOT stale — that is coverage's business", () => {
|
||||
const stale = findStaleTranslations({
|
||||
baseEn: { m: { k: "old english" } },
|
||||
headEn: { m: { k: "new english" } },
|
||||
baseLocales: { de: {} },
|
||||
headLocales: { de: {} },
|
||||
});
|
||||
|
||||
assert.deepEqual(stale, []);
|
||||
});
|
||||
|
||||
test("a BRAND-NEW EN key cannot be stale anywhere", () => {
|
||||
const stale = findStaleTranslations({
|
||||
baseEn: {},
|
||||
headEn: { m: { k: "brand new" } },
|
||||
baseLocales: { "pt-BR": {} },
|
||||
headLocales: { "pt-BR": { m: { k: "tradução nova" } } },
|
||||
});
|
||||
|
||||
assert.deepEqual(stale, []);
|
||||
});
|
||||
|
||||
test("an unchanged EN value never flags, however stale the translation actually is", () => {
|
||||
// This is the deliberate "freeze existing debt" semantics: a diff-aware gate cannot
|
||||
// know what a pre-existing translation was derived from, so it only judges what the
|
||||
// current change touches. Pre-existing drift stays invisible instead of failing every
|
||||
// unrelated PR.
|
||||
const stale = findStaleTranslations({
|
||||
baseEn: { m: { k: "same english" } },
|
||||
headEn: { m: { k: "same english" } },
|
||||
baseLocales: { "pt-BR": { m: { k: "tradução obsoleta de outra era" } } },
|
||||
headLocales: { "pt-BR": { m: { k: "tradução obsoleta de outra era" } } },
|
||||
});
|
||||
|
||||
assert.deepEqual(stale, []);
|
||||
});
|
||||
|
||||
test("a renamed key (delete + add) flags nothing — the fix pattern used by #8463", () => {
|
||||
const stale = findStaleTranslations({
|
||||
baseEn: { m: { oldName: "old english" } },
|
||||
headEn: { m: { newName: "new english" } },
|
||||
baseLocales: { "pt-BR": { m: { oldName: "português antigo" } } },
|
||||
headLocales: { "pt-BR": { m: { newName: "__MISSING__:new english" } } },
|
||||
});
|
||||
|
||||
assert.deepEqual(stale, []);
|
||||
});
|
||||
|
||||
test("every affected locale is reported, and en itself is never a target", () => {
|
||||
const stale = findStaleTranslations({
|
||||
baseEn: { m: { k: "old" } },
|
||||
headEn: { m: { k: "new" } },
|
||||
baseLocales: {
|
||||
en: { m: { k: "old" } },
|
||||
"pt-BR": { m: { k: "antigo" } },
|
||||
de: { m: { k: "alt" } },
|
||||
fr: { m: { k: "__MISSING__:old" } },
|
||||
},
|
||||
headLocales: {
|
||||
en: { m: { k: "new" } },
|
||||
"pt-BR": { m: { k: "antigo" } },
|
||||
de: { m: { k: "alt" } },
|
||||
fr: { m: { k: "__MISSING__:old" } },
|
||||
},
|
||||
});
|
||||
|
||||
assert.deepEqual(
|
||||
stale.map((s) => s.locale).sort(),
|
||||
["de", "pt-BR"],
|
||||
"en must be excluded and __MISSING__ locales skipped"
|
||||
);
|
||||
});
|
||||
|
||||
test("a value that only changes whitespace still counts as an edit", () => {
|
||||
// Conservative on purpose: trailing-space churn is rare, and treating it as a no-op
|
||||
// would let a real reword slip through behind an innocuous-looking diff.
|
||||
const stale = findStaleTranslations({
|
||||
baseEn: { m: { k: "hello" } },
|
||||
headEn: { m: { k: "hello " } },
|
||||
baseLocales: { de: { m: { k: "hallo" } } },
|
||||
headLocales: { de: { m: { k: "hallo" } } },
|
||||
});
|
||||
|
||||
assert.deepEqual(stale, [{ key: "m.k", locale: "de" }]);
|
||||
});
|
||||
|
||||
test("non-string leaves (numbers, booleans, null) are compared without throwing", () => {
|
||||
const stale = findStaleTranslations({
|
||||
baseEn: { m: { n: 1, b: true, z: null } },
|
||||
headEn: { m: { n: 2, b: true, z: null } },
|
||||
baseLocales: { de: { m: { n: 1, b: true, z: null } } },
|
||||
headLocales: { de: { m: { n: 1, b: true, z: null } } },
|
||||
});
|
||||
|
||||
assert.deepEqual(stale, [{ key: "m.n", locale: "de" }]);
|
||||
});
|
||||
Reference in New Issue
Block a user