mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-04 06:12:10 +03:00
* fix(ci): stop the reconciliation range and the fragment sweep from hiding work Two release-tooling defects found during the v3.8.49 run (gaps 4 and 7 of the process dossier). Both fail by hiding work rather than announcing themselves, which is why each one had already cost a real mistake. ## The reconciliation range was 62× too wide `list-uncovered-commits.mjs` bounded its scan with `git describe --tags`. Releases reach `main` by SQUASH, so no commit on a release branch is ever an ancestor of the tag, and `vPREV..HEAD` re-lists the un-squashed history of every earlier cycle. Measured on release/v3.8.50 @7eca04fd12: v3.8.49..HEAD ....... 1361 commits cycle open..HEAD ..... 22 commits The report drowns in noise, and that is how a previous reconciliation let ~200 PRs through with no CHANGELOG bullet. The base is now resolved by CONTENT — the oldest commit that introduced this version string into package.json — deliberately NOT by commit subject, because the subject has already changed format once: chore(release): bump v3.8.49 (development cycle version) older chore(release): open v3.8.50 development cycle current A message-matching resolver would have silently reverted to the broken tag base the first time someone reworded the bump. The fallback now writes a WARNING to stderr explaining that a tag range re-lists previous cycles, so a shallow clone degrades loudly instead of quietly reproducing the bug. One of the five tests asserts exactly that the warning says "squash" and "noise". ## The back-merge resurrects fragments that already shipped The release lands on `main` as one squash commit, so `main` still carries every `changelog.d/` fragment the reconciliation folded in and deleted. Back-merging `main` restores all of them — 191 in the v3.8.49 run. Nothing breaks at that instant; the next aggregation folds them in a SECOND time and the section grows duplicates that have to be hand-unpicked. New `scripts/release/sweep-stale-fragments.mjs` (`npm run sweep:stale-fragments`) reports them, and `--apply` removes them. Report mode exits 1 so the back-merge step can gate on it. The identity rule took two attempts, and the second one exists because running the script against the live repo refuted the first. Matching on any `#N` in the bullet flagged `changelog.d/features/8980-deprecate-gemini-cli-provider.md` as stale, because that bullet cites issue **#7034** for context and #7034 shipped in an earlier cycle — it would have deleted an unreleased fragment and dropped its credit. A bullet routinely cites issues it merely references; only the `<PR-number>-<slug>.md` filename says which PR the fragment *is*. That case is now a regression test. Every ambiguous case resolves toward KEEPING: no number in the filename falls back to normalized text, text shorter than 12 chars is never matched, and anything matching neither is kept. A surviving duplicate is a nuisance someone notices; a deleted fragment silently costs a contributor their credit. node --import tsx/esm --test tests/unit/release-cycle-base-resolver.test.ts # 5 pass node --import tsx/esm --test tests/unit/sweep-stale-fragments.test.ts # 11 pass node scripts/release/list-uncovered-commits.mjs --json → baseed2db6cb19, baseSource "cycle-open", total 22 (was 1361) node scripts/release/sweep-stale-fragments.mjs → 4 fragments, 0 stale, exit 0 * docs(changelog): fragment for #8985 * fix(ci): four quality gates that punished the wrong thing Gaps 6, 9, 10 and 23 of the v3.8.49 process dossier. Each one either blocked work it should have waved through, or reported a number that was never the code's. ## 6 — test-masking is unusable at release scale My own dossier entry for this was WRONG and the measurement says so: tracked test files ............ 3977 (I had written 1277) absolute tautology scan ....... ~1 s (I had written >30 min) the diff uses base...HEAD three dots — already merge-base diff vs release branch ........ 0 files, 0 s diff vs main (today) .......... 3 files, 0 s The base choice was never the problem, and it cannot be reproduced today at all: `main` has since received the v3.8.49 squash, so the merge-base is recent. The pathology only exists DURING a release, in the window before `main` gets the squash — then the merge-base is the PREVIOUS cycle's fork point and the diff legitimately spans the whole cycle (~1277 changed test files, each costing a `git show` process plus a full regex pass). That is the same squash-merge topology as gap 4, and it is why the check ran twice without finishing. Fix: above 300 changed test files the per-file diff subchecks are skipped, since every one of those files was already gated by this check on its own PR. The absolute tautology scan still runs unconditionally over all 3977 files, so the floor is untouched. The skip is deliberately loud — a silent skip is gap 12, which cost two production bugs this cycle. `shouldSkipDiffSubchecks` never skips on unparseable input, so a broken count cannot disable the gate. ## 9 — a capital letter invalidated 41 translations `"Reset Defaults"` → `"Reset defaults"` marked the key stale in 41 locales. Every translation was still correct, and in locales with no letter case the "fix" is not expressible. Worse, the escape hatch (`__MISSING__:`) is BANNED in `vi` by tests/unit/i18n-vi-completeness.test.ts, so `vi` had no legitimate way out. `isCosmeticRewrite` folds case, whitespace runs and trailing punctuation — and nothing else. Most of the nine tests exist to pin what is NOT cosmetic: a changed word, an added word, and any edit inside an interpolation like `{count}` all still flag. Two end-to-end tests hold both directions: a cosmetic edit leaves every locale alone, a real rewrite still flags all of them. ## 10 — the ratchet compared numbers from two different auditors `pipx install zizmor` was unpinned, so the runner installed whatever PyPI served that day and measured 1 finding MORE than the devbox on the identical commit (190 vs 189) — a second rebaseline push per release, chasing a number that was never the code's. Pinned to 1.25.2 (what the devbox runs), and check-workflows.mjs now prints `zizmorVersion=` next to the count so any future rebaseline is traceable to the tool that produced it. ## 23 — a PR pointed at its own branch #8912 has head == base == release/v3.8.50: no diff, can never merge, and it sits in the queue with a full check board on every push to that branch. It survived because nothing looks wrong — the checks pass, since there is nothing to check. New guard in the `changes` job (one field comparison, before anything is spent). The distinction that makes it safe to block on: an equal head/base BRANCH is conclusive, an equal head/base SHA is NOT — a branch cut moments ago has an identical tip and is legitimate, so that case warns instead of failing. Half a signal never fails either. node --import tsx/esm --test tests/unit/test-masking-release-scale.test.ts # 6 pass node --import tsx/esm --test tests/unit/ui-value-drift-cosmetic.test.ts # 9 pass node --import tsx/esm --test tests/unit/pr-self-target-guard.test.ts # 7 pass check:workflows --ratchet → 178 findings, zizmorVersion=zizmor 1.25.2, baseline 190 the i18n suite is unaffected (5 files re-run, all green) * fix(ci): allowlist the four CI-only env vars the new gates read The env-doc-sync gate failed three unit shards plus Docs Gates on this PR, and it was right to: it requires every `process.env.X` read in code to be documented in `.env.example`, and this PR introduced four new reads. They do not belong in `.env.example`. That file is OmniRoute's runtime configuration; these are CI signals with no meaning in a user's `.env`: HEAD_REF / HEAD_SHA / BASE_SHA the `changes` job passes github.head_ref, github.base_ref and the PR head/base SHAs to the self-targeting-PR guard TEST_MASKING_MAX_CHANGED_TESTS the escape hatch that raises the test-masking gate's release-scale skip threshold So they go in IGNORE_FROM_CODE, which exists for exactly this and already carries the precedent one line above: `BASE_REF`, allowlisted because CI passes it to the OpenAPI breaking-change gate. `BASE_REF` being already listed is also why only four of my five reads failed. Each entry carries its justification and the script that reads it, per the allowlist policy. node --import tsx/esm --test tests/unit/issue-7793-env-doc-sync-repro.test.ts # 1 pass npm run check:env-doc-sync → all three directions in sync * fix(i18n): narrow the cosmetic-rewrite exemption to the scope actually reported The gap-9 fix folded whitespace in addition to case, and that collided with a pre-existing test which pins the opposite — tests/unit/i18n-ui-value-drift.test.ts, "a value that only changes whitespace still counts as an edit". Its comment states the reasoning: Conservative on purpose: trailing-space churn is rare, and treating it as a no-op would let a real reword slip through behind an innocuous-looking diff. That is a documented decision by whoever wrote it. The problem actually reported was CASE — `"Reset Defaults"` → `"Reset defaults"` invalidating 41 correct translations — and whitespace was scope I added on my own. Reversing someone else's reasoned call, silently, to fix something nobody reported is not this change's job, so the exemption is narrowed to case + trailing terminal punctuation. No test pins either of those. The reported case is still fixed, verified end to end: that rewrite invalidates 0 locales. And whitespace is now asserted NON-cosmetic in my own test file too, so a later tidy-up cannot quietly fold it back in. node --import tsx/esm --test tests/unit/i18n-ui-value-drift.test.ts # 11 pass (pre-existing) node --import tsx/esm --test tests/unit/ui-value-drift-cosmetic.test.ts # 10 pass --------- Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
650 lines
27 KiB
JavaScript
650 lines
27 KiB
JavaScript
#!/usr/bin/env node
|
|
// scripts/check/check-test-masking.mjs
|
|
// Gate anti test-masking (a preocupação nº1 do CLAUDE.md: "subagente não pode
|
|
// enfraquecer/remover asserts pra ficar verde"). Para cada arquivo de teste MODIFICADO
|
|
// num PR, compara a contagem de asserts base vs HEAD: sinaliza REMOÇÃO LÍQUIDA de asserts
|
|
// e NOVAS tautologias `assert.ok(true)`. Heurístico mas alto-sinal. Espelha o plumbing
|
|
// de check-pr-test-policy.mjs (diff base...HEAD); no-op fora de contexto de PR.
|
|
//
|
|
// v2 (6A.10): acrescenta 3 novos subchecks:
|
|
// 1. Arquivos de teste DELETADOS: --diff-filter=MDR com detecção de rename.
|
|
// 2. Aumento líquido de .skip/.todo/.only/{skip:true}: esconde asserts sem remover.
|
|
// 3. Tautologias extras: expect(true).toBe(true), assert.equal(1,1), assert.ok(true).
|
|
import fs from "node:fs";
|
|
import { execFileSync } from "node:child_process";
|
|
import { pathToFileURL } from "node:url";
|
|
|
|
const TEST_RE = /\.(test|spec)\.(ts|tsx)$/;
|
|
// Production TypeScript sources (excludes test files, handled separately via TEST_RE).
|
|
const PROD_SRC_RE = /\.(ts|tsx|mts|cts)$/;
|
|
|
|
/** Conta chamadas de assert.*( / assert( / expect( . */
|
|
export function countAssertions(src) {
|
|
const a = (src.match(/\bassert\b\s*[.(]/g) || []).length;
|
|
const e = (src.match(/\bexpect\s*\(/g) || []).length;
|
|
return a + e;
|
|
}
|
|
|
|
/** Conta tautologias assert.ok(true). */
|
|
export function countTautologies(src) {
|
|
return (src.match(/\bassert\s*\.\s*ok\s*\(\s*true\s*\)/g) || []).length;
|
|
}
|
|
|
|
/**
|
|
* (6A.10 subcheck 2) Conta marcadores de skip/todo/only que silenciam testes:
|
|
* - .skip(, .todo(, .only( — em qualquer runner (node:test, jest, vitest)
|
|
* - { skip: true } — opção de objeto node:test
|
|
*/
|
|
export function countSkips(src) {
|
|
const modifiers = (src.match(/\.\s*(?:skip|todo|only)\s*\(/g) || []).length;
|
|
const skipOpt = (src.match(/\{\s*skip\s*:\s*true\s*\}/g) || []).length;
|
|
return modifiers + skipOpt;
|
|
}
|
|
|
|
/**
|
|
* (6A.10 subcheck 3) Conta tautologias que mantêm os asserts no texto mas nunca
|
|
* verificam nada real:
|
|
* - expect(true).toBe(true)
|
|
* - assert.equal(1, 1) / assert.strictEqual(1, 1)
|
|
* - assert.ok(true) (já coberto por countTautologies; incluído aqui para completude)
|
|
*/
|
|
export function countExtendedTautologies(src) {
|
|
let count = 0;
|
|
// expect(true).toBe(true)
|
|
count += (src.match(/\bexpect\s*\(\s*true\s*\)\s*\.\s*toBe\s*\(\s*true\s*\)/g) || []).length;
|
|
// assert.equal(1, 1) / assert.strictEqual(1, 1) — literal numeric identity
|
|
count += (src.match(/\bassert\s*\.\s*(?:strict)?[Ee]qual\s*\(\s*1\s*,\s*1\s*\)/g) || []).length;
|
|
// assert.ok(true)
|
|
count += (src.match(/\bassert\s*\.\s*ok\s*\(\s*true\s*\)/g) || []).length;
|
|
return count;
|
|
}
|
|
|
|
/**
|
|
* (#6404) Narrower sibling of countExtendedTautologies(), deliberately EXCLUDING
|
|
* `assert.ok(true)`: that pattern is intentionally left to the lenient, diff-only,
|
|
* new-occurrences-only subcheck 3 above, because ~15 pre-existing, verified-legitimate
|
|
* uses already exist repo-wide (documented fallbacks like "expected to throw" /
|
|
* "DB not available, expected" in try/catch branches) — an absolute, always-on scan
|
|
* against all of them would be a mass false-positive, not a real signal.
|
|
*
|
|
* `expect(true).toBe(true)` / `assert.equal(1, 1)` / `assert.strictEqual(1, 1)` have
|
|
* no such legitimate use anywhere in this codebase (verified zero pre-existing hits
|
|
* after fixing #6404's playground-api-tab.test.tsx) — a genuinely bare, no-argument
|
|
* tautology is never a deliberate pattern here, so it is safe to fail on ANY hit,
|
|
* with or without a PR diff to compare against. See scanBareTautologies() below.
|
|
*/
|
|
export function countBareTautologies(src) {
|
|
let count = 0;
|
|
// expect(true).toBe(true)
|
|
count += (src.match(/\bexpect\s*\(\s*true\s*\)\s*\.\s*toBe\s*\(\s*true\s*\)/g) || []).length;
|
|
// assert.equal(1, 1) / assert.strictEqual(1, 1) — literal numeric identity
|
|
count += (src.match(/\bassert\s*\.\s*(?:strict)?[Ee]qual\s*\(\s*1\s*,\s*1\s*\)/g) || []).length;
|
|
return count;
|
|
}
|
|
|
|
// ─── (6348) Subcheck 4: inline-reimplemented prod conditions (REPORT-ONLY) ───
|
|
// A test that copies a conditional expression out of production code (instead of
|
|
// importing and exercising the symbol that owns it) is the wrong-shape-contract-test
|
|
// class (#6216): the assertion re-encodes the branch locally, so it stays green even
|
|
// when the real prod condition drifts. This subcheck is a HEURISTIC, textual gate
|
|
// mirroring the count* helpers above — it never parses an AST. It is REPORT-ONLY for
|
|
// now (warns, does not fail the gate).
|
|
|
|
/** Collapse all runs of whitespace to a single space and trim. */
|
|
function normalizeWhitespace(s) {
|
|
return (s || "").replace(/\s+/g, " ").trim();
|
|
}
|
|
|
|
/**
|
|
* Count "significant" tokens in a normalized condition. Single-char identifiers
|
|
* (`x`, `i`) and single-digit numeric literals (`0`, `1`) are treated as noise and
|
|
* NOT counted; operators and multi-char identifiers/numbers ARE. This is what makes
|
|
* `x > 0` trivial (1 significant token) while `status >= 500` is meaningful (3):
|
|
* - `status >= 500` → status, >=, 500 → 3
|
|
* - `x === LIMIT && y` → ===, LIMIT, && → 3
|
|
* - `x > 0` → > → 1
|
|
*/
|
|
export function countSignificantTokens(cond) {
|
|
const tokens =
|
|
(cond || "").match(
|
|
/===|!==|==|!=|>=|<=|&&|\|\||[<>+\-*/%!]|[A-Za-z_$][\w$]*|\d+(?:\.\d+)?/g
|
|
) || [];
|
|
let count = 0;
|
|
for (const tk of tokens) {
|
|
if (/^[A-Za-z_$]/.test(tk)) {
|
|
if (tk.length >= 2) count++; // multi-char identifier
|
|
} else if (/^\d/.test(tk)) {
|
|
if (tk.length >= 2) count++; // multi-digit number
|
|
} else {
|
|
count++; // operator
|
|
}
|
|
}
|
|
return count;
|
|
}
|
|
|
|
/** A condition is "meaningful" when it carries ≥3 significant tokens. */
|
|
function isSignificantCondition(cond) {
|
|
return countSignificantTokens(cond) >= 3;
|
|
}
|
|
|
|
/**
|
|
* Extract meaningful (≥3-token) conditional expressions from a production source,
|
|
* paired with the nearest enclosing declared symbol that "owns" them. Covers
|
|
* `if (...)` (via paren balancing) and comparison-bearing ternaries (`a === b ? … : …`).
|
|
* Returns [{ condition (whitespace-normalized), owner }].
|
|
*/
|
|
export function extractProdConditions(src) {
|
|
const results = [];
|
|
if (!src) return results;
|
|
|
|
// Declarations (function / const / let / var) with their positions, so each
|
|
// condition can be attributed to the symbol whose body it lives in.
|
|
const decls = [];
|
|
const declRe =
|
|
/(?:export\s+)?(?:default\s+)?(?:async\s+)?function\s+([A-Za-z_$][\w$]*)|(?:export\s+)?(?:const|let|var)\s+([A-Za-z_$][\w$]*)\s*=/g;
|
|
let dm;
|
|
while ((dm = declRe.exec(src))) {
|
|
decls.push({ index: dm.index, name: dm[1] || dm[2] });
|
|
}
|
|
const ownerAt = (idx) => {
|
|
let owner = "";
|
|
for (const d of decls) {
|
|
if (d.index <= idx) owner = d.name;
|
|
else break;
|
|
}
|
|
return owner;
|
|
};
|
|
|
|
const seen = new Set();
|
|
const pushCond = (raw, owner) => {
|
|
const norm = normalizeWhitespace(raw);
|
|
if (!norm || seen.has(norm) || !isSignificantCondition(norm)) return;
|
|
seen.add(norm);
|
|
results.push({ condition: norm, owner });
|
|
};
|
|
|
|
// if (...) — balance parentheses to capture the full condition.
|
|
const ifRe = /\bif\s*\(/g;
|
|
let m;
|
|
while ((m = ifRe.exec(src))) {
|
|
let depth = 1;
|
|
let i = m.index + m[0].length;
|
|
for (; i < src.length && depth > 0; i++) {
|
|
const ch = src[i];
|
|
if (ch === "(") depth++;
|
|
else if (ch === ")") depth--;
|
|
}
|
|
pushCond(src.slice(m.index + m[0].length, i - 1), ownerAt(m.index));
|
|
}
|
|
|
|
// Comparison-bearing ternaries: `<lhs> <cmp> <rhs> ? … : …` (best-effort, low-noise).
|
|
const ternRe =
|
|
/([A-Za-z_$][\w$).\]]*\s*(?:===|!==|==|!=|>=|<=|>|<)\s*[^?;{}\n]+?)\s*\?/g;
|
|
let t;
|
|
while ((t = ternRe.exec(src))) {
|
|
pushCond(t[1], ownerAt(t.index));
|
|
}
|
|
|
|
return results;
|
|
}
|
|
|
|
/**
|
|
* Collect the identifiers/module specifiers a test file imports, so we can tell
|
|
* whether it exercises a prod symbol through the real import (clean) or merely
|
|
* re-implements one of its conditions locally (masked). Returns a Set of names:
|
|
* imported bindings, module paths, and module basenames.
|
|
*/
|
|
export function extractImports(src) {
|
|
const names = new Set();
|
|
if (!src) return names;
|
|
const addModule = (mod) => {
|
|
names.add(mod);
|
|
const base = mod.split("/").pop().replace(/\.\w+$/, "");
|
|
if (base) names.add(base);
|
|
};
|
|
let m;
|
|
const importRe = /import\s+(?:type\s+)?([^;]*?)\s+from\s+['"]([^'"]+)['"]/g;
|
|
while ((m = importRe.exec(src))) {
|
|
addModule(m[2]);
|
|
for (const id of m[1].match(/[A-Za-z_$][\w$]*/g) || []) {
|
|
if (id !== "as" && id !== "type") names.add(id);
|
|
}
|
|
}
|
|
const dynRe = /import\s*\(\s*['"]([^'"]+)['"]\s*\)/g;
|
|
while ((m = dynRe.exec(src))) addModule(m[1]);
|
|
const reqRe = /require\s*\(\s*['"]([^'"]+)['"]\s*\)/g;
|
|
while ((m = reqRe.exec(src))) addModule(m[1]);
|
|
return names;
|
|
}
|
|
|
|
/**
|
|
* PURE core of subcheck 4. Given the sources of the prod files changed in a PR,
|
|
* one test file's source, and the set of names that test imports: return the prod
|
|
* conditions the test re-implements textually WITHOUT importing the symbol that owns
|
|
* them. Whitespace is squashed on both sides so spacing differences never mask a hit.
|
|
* Returns [{ condition, owner }].
|
|
*/
|
|
export function findReimplementedConditions(prodSources, testSource, testImports) {
|
|
const flags = [];
|
|
if (!testSource) return flags;
|
|
const imports =
|
|
testImports instanceof Set ? testImports : new Set(testImports || []);
|
|
const squash = (s) => (s || "").replace(/\s+/g, "");
|
|
const testSq = squash(testSource);
|
|
const seen = new Set();
|
|
for (const prod of prodSources || []) {
|
|
for (const { condition, owner } of extractProdConditions(prod)) {
|
|
if (owner && imports.has(owner)) continue; // exercised through the real import
|
|
if (seen.has(condition)) continue;
|
|
if (testSq.includes(squash(condition))) {
|
|
seen.add(condition);
|
|
flags.push({ condition, owner: owner || null });
|
|
}
|
|
}
|
|
}
|
|
return flags;
|
|
}
|
|
|
|
/**
|
|
* (6A.10 subcheck 1) Sinaliza arquivos de teste DELETADOS ou renomeados-e-não-
|
|
* substituídos. Recebe lista de paths de arquivos de teste que foram deletados
|
|
* (filtro D do git diff --diff-filter=MDR).
|
|
*
|
|
* `deletionAllowlist` (`_deletedWithReplacement` no test-masking-allowlist.json)
|
|
* isenta uma deleção SOMENTE quando o substituto declarado existe no HEAD e é
|
|
* ele próprio um arquivo de teste — o caso "reescrito em outro path sem rename
|
|
* detectável" (conteúdo novo demais para o -M do git). Qualquer entrada cujo
|
|
* substituto não exista ou não seja teste continua flagada.
|
|
*/
|
|
export function evaluateDeletedFiles(
|
|
deletedPaths,
|
|
deletionAllowlist = {},
|
|
fileExists = fs.existsSync
|
|
) {
|
|
const flags = [];
|
|
for (const f of deletedPaths) {
|
|
if (!TEST_RE.test(f)) continue;
|
|
const entry = deletionAllowlist[f];
|
|
if (entry && typeof entry.replacement === "string") {
|
|
if (TEST_RE.test(entry.replacement) && fileExists(entry.replacement)) continue;
|
|
flags.push(
|
|
`${f}: deleção allowlistada mas o substituto declarado (${entry.replacement}) não existe ou não é arquivo de teste`
|
|
);
|
|
continue;
|
|
}
|
|
flags.push(
|
|
`${f}: arquivo de teste deletado — revisão humana obrigatória (mascaramento alto-sinal)`
|
|
);
|
|
}
|
|
return flags;
|
|
}
|
|
|
|
/**
|
|
* Parse `git diff --name-status -M --diff-filter=DR` output, separating TRUE
|
|
* test-file deletions ("D\tpath") from RENAMES ("R<score>\told\tnew").
|
|
*
|
|
* A rename whose destination is still a test file is a *relocation* (the test
|
|
* was substituted at a new path, not removed) — per this file's subcheck-1
|
|
* contract it must NOT be treated as a deletion; the assert-reduction check
|
|
* still runs across the rename to catch gutting-via-rename. A rename that lands
|
|
* OUTSIDE test scope (test → non-test) removes the test and is treated as a
|
|
* deletion. Returns test-file paths only.
|
|
*/
|
|
export function partitionDeletedRenamed(nameStatusOutput) {
|
|
const deletedTests = [];
|
|
const renames = [];
|
|
for (const line of (nameStatusOutput || "").split("\n")) {
|
|
if (!line.trim()) continue;
|
|
const parts = line.split("\t").map((s) => s.trim());
|
|
const status = parts[0] || "";
|
|
if (status.startsWith("D")) {
|
|
if (TEST_RE.test(parts[1] || "")) deletedTests.push(parts[1]);
|
|
} else if (status.startsWith("R")) {
|
|
const from = parts[1] || "";
|
|
const to = parts[2] || "";
|
|
if (TEST_RE.test(from)) renames.push({ from, to });
|
|
}
|
|
}
|
|
return { deletedTests, renames };
|
|
}
|
|
|
|
/**
|
|
* Avalia por-arquivo: flag em remoção líquida de asserts, nova tautologia,
|
|
* aumento líquido de skips, ou nova tautologia extendida.
|
|
*
|
|
* Cada entrada de perFile deve ter:
|
|
* { file, baseAsserts, headAsserts, baseTaut, headTaut,
|
|
* baseSkips, headSkips, baseExtTaut, headExtTaut }
|
|
*
|
|
* Os campos de skip e extTaut são opcionais (default 0) para compatibilidade
|
|
* com chamadas legadas que só passam baseAsserts/headAsserts/baseTaut/headTaut.
|
|
*/
|
|
/**
|
|
* (#6634) `check-test-masking.test.ts` legitimately embeds tautology-pattern string
|
|
* literals (`assert.ok(true)`, `expect(true).toBe(true)`, `assert.equal(1,1)`) as
|
|
* FIXTURES to exercise `countBareTautologies()`/`scanBareTautologies()` (#6404). The
|
|
* diff-based tautology counters (`countTautologies()`/`countExtendedTautologies()`)
|
|
* are dumb regex scans of raw source text with no awareness that a literal sits
|
|
* inside a fixture string rather than real assertion code, so any new fixture line
|
|
* self-trips a HARD "new tautology" flag on the gate's own regression-test file.
|
|
* Mirrors the exclusion `scanBareTautologies()` already applies for the same reason.
|
|
*
|
|
* The exclusion covers the whole `check-test-masking*` gate self-test family — not
|
|
* just `check-test-masking.test.ts` but sibling regression files such as
|
|
* `check-test-masking-selfref-6634.test.ts`, which likewise embed tautology-pattern
|
|
* literals as fixtures/documentation to prove this gate's own behavior.
|
|
*/
|
|
const SELF_TEST_FIXTURE_RE = /(^|\/)check-test-masking(-[\w-]+)?\.test\.tsx?$/;
|
|
function isSelfTestFixtureFile(file) {
|
|
return SELF_TEST_FIXTURE_RE.test(file);
|
|
}
|
|
|
|
export function evaluateMasking(perFile, assertReductionAllowlist = new Set()) {
|
|
const flags = [];
|
|
for (const f of perFile) {
|
|
const baseSkips = f.baseSkips ?? 0;
|
|
const headSkips = f.headSkips ?? 0;
|
|
const baseExtTaut = f.baseExtTaut ?? 0;
|
|
const headExtTaut = f.headExtTaut ?? 0;
|
|
const isSelfTestFixture = isSelfTestFixtureFile(f.file);
|
|
|
|
// The net-assert-REDUCTION signal can be allowlisted per file when the reduction is a
|
|
// verified-legitimate refactor/field-removal (config/quality/test-masking-allowlist.json).
|
|
// The tautology / skip / deletion signals below are NEVER allowlisted.
|
|
if (f.headAsserts < f.baseAsserts && !assertReductionAllowlist.has(f.file))
|
|
flags.push(
|
|
`${f.file}: asserts ${f.baseAsserts} → ${f.headAsserts} (REMOÇÃO de ${f.baseAsserts - f.headAsserts} — enfraquecimento?)`
|
|
);
|
|
if (!isSelfTestFixture && f.headTaut > f.baseTaut)
|
|
flags.push(`${f.file}: nova(s) ${f.headTaut - f.baseTaut} tautologia(s) assert.ok(true)`);
|
|
if (headSkips > baseSkips)
|
|
flags.push(
|
|
`${f.file}: ${headSkips - baseSkips} novo(s) .skip/.todo/.only (asserts silenciados sem remoção)`
|
|
);
|
|
if (!isSelfTestFixture && headExtTaut > baseExtTaut)
|
|
flags.push(
|
|
`${f.file}: nova(s) ${headExtTaut - baseExtTaut} tautologia(s) estendida(s) (expect(true).toBe(true) / assert.equal(1,1))`
|
|
);
|
|
}
|
|
return flags;
|
|
}
|
|
|
|
/**
|
|
* (#6404) Absolute floor scan for bare tautologies (`expect(true).toBe(true)`,
|
|
* `assert.equal(1, 1)` / `assert.strictEqual(1, 1)`), independent of PR diffing.
|
|
*
|
|
* The subcheck-3 diff logic above (`evaluateMasking`'s `headExtTaut > baseExtTaut`)
|
|
* only fires for a tautology INTRODUCED within the current PR's own diff, and
|
|
* `resolveBase()` returns `null` outside CI (no `GITHUB_BASE_SHA`/`GITHUB_BASE_REF`),
|
|
* so a local `npm run check:test-masking` run silently no-ops — "sem base ref —
|
|
* pulando" — regardless of what the tests actually contain. That is exactly how
|
|
* #6404's `expect(true).toBe(true)` in `playground-api-tab.test.tsx` slipped through
|
|
* for a full release cycle after merging once (the diff-only gate has nothing to
|
|
* compare a pre-existing, already-merged tautology against, and local runs never
|
|
* scan repo content at all). This scans every tracked test file's current content,
|
|
* in or out of PR context, so a stray tautology can never hide once merged.
|
|
*
|
|
* Uses `countBareTautologies()` (not `countExtendedTautologies()`) — deliberately
|
|
* excludes `assert.ok(true)`, which has ~15 verified-legitimate pre-existing uses
|
|
* repo-wide and stays governed by the lenient, new-occurrence-only diff subcheck.
|
|
*
|
|
* `check-test-masking.test.ts` is excluded — its fixtures legitimately embed the
|
|
* literal pattern as string literals to exercise the count* helpers themselves.
|
|
*/
|
|
export function scanBareTautologies(testFiles, readFile) {
|
|
const read = readFile || ((f) => fs.readFileSync(f, "utf8"));
|
|
const flags = [];
|
|
for (const file of testFiles || []) {
|
|
if (isSelfTestFixtureFile(file)) continue;
|
|
let src;
|
|
try {
|
|
src = read(file);
|
|
} catch {
|
|
continue;
|
|
}
|
|
const count = countBareTautologies(src);
|
|
if (count > 0) {
|
|
flags.push(
|
|
`${file}: ${count} tautologia(s) pura(s) (expect(true).toBe(true) / assert.equal(1,1)) — ` +
|
|
"substitua por um assert real do comportamento observável"
|
|
);
|
|
}
|
|
}
|
|
return flags;
|
|
}
|
|
|
|
function git(args) {
|
|
try {
|
|
return execFileSync("git", args, { encoding: "utf8" });
|
|
} catch {
|
|
return "";
|
|
}
|
|
}
|
|
|
|
/** All git-tracked test files (`.test.ts(x)`/`.spec.ts(x)`), repo-wide — used by the
|
|
* absolute floor scan so it also covers files untouched by the current diff/PR. */
|
|
function listTrackedTestFiles() {
|
|
return git(["ls-files"])
|
|
.split("\n")
|
|
.map((s) => s.trim())
|
|
.filter((f) => TEST_RE.test(f));
|
|
}
|
|
|
|
function resolveBase() {
|
|
if (process.env.GITHUB_BASE_SHA) return process.env.GITHUB_BASE_SHA;
|
|
if (process.env.GITHUB_BASE_REF) return `origin/${process.env.GITHUB_BASE_REF}`;
|
|
return null;
|
|
}
|
|
|
|
/**
|
|
* Whether the per-file diff subchecks should be skipped for being too large to be a
|
|
* reviewable unit. Exported so the threshold behavior is testable without a repo: the
|
|
* boundary is what matters, and an off-by-one here either blocks a release or silently
|
|
* disables the check on a big-but-legitimate PR.
|
|
*
|
|
* `max <= 0` disables the skip entirely (always analyze) — a deliberate escape hatch.
|
|
*/
|
|
export function shouldSkipDiffSubchecks(changedCount, max) {
|
|
const n = Number(changedCount);
|
|
const cap = Number(max);
|
|
if (!Number.isFinite(n) || n < 0) return false;
|
|
if (!Number.isFinite(cap) || cap <= 0) return false;
|
|
return n > cap;
|
|
}
|
|
|
|
function main() {
|
|
// (#6404) Absolute floor scan — runs unconditionally, PR or not, so a tautology
|
|
// that is already merged into the base (and thus invisible to the diff-only
|
|
// subchecks below) or a local pre-push run (which has no PR base to diff
|
|
// against) still gets caught. See scanBareTautologies() doc comment.
|
|
let bareTautAllowlist = new Set();
|
|
try {
|
|
const raw = JSON.parse(fs.readFileSync("config/quality/test-masking-allowlist.json", "utf8"));
|
|
bareTautAllowlist = new Set(raw._bareTautologyAllowlist || []);
|
|
} catch {
|
|
// no allowlist file — treat as empty
|
|
}
|
|
const trackedTestFiles = listTrackedTestFiles().filter((f) => !bareTautAllowlist.has(f));
|
|
const absoluteTautFlags = scanBareTautologies(trackedTestFiles);
|
|
if (absoluteTautFlags.length) {
|
|
console.error(
|
|
`[test-masking] ${absoluteTautFlags.length} tautologia(s) pura(s) encontradas ` +
|
|
`(scan absoluto — roda com ou sem contexto de PR):\n` +
|
|
absoluteTautFlags.map((f) => " ✗ " + f).join("\n") +
|
|
`\n → substitua por um assert real do comportamento observável.`
|
|
);
|
|
process.exit(1);
|
|
}
|
|
|
|
const base = resolveBase();
|
|
if (!base) {
|
|
console.log(
|
|
"[test-masking] sem base ref (não é PR) — pulando checks de diff (scan absoluto de tautologias OK)."
|
|
);
|
|
return;
|
|
}
|
|
|
|
// (6A.10 subcheck 1) Arquivos de teste deletados/renomeados via MDR filter.
|
|
// Renames test→test são RELOCAÇÕES (substituição) e passam pela verificação de
|
|
// redução de asserts abaixo (gutting-via-rename ainda flaga); só deleções reais
|
|
// e renames test→não-teste contam como remoção de teste.
|
|
const { deletedTests, renames } = partitionDeletedRenamed(
|
|
git(["diff", "--name-status", "-M", "--diff-filter=DR", `${base}...HEAD`])
|
|
);
|
|
|
|
const relocatedOutOfTest = [];
|
|
const renamePerFile = [];
|
|
for (const { from, to } of renames) {
|
|
if (!TEST_RE.test(to)) {
|
|
// test → non-test: the test was removed from coverage.
|
|
relocatedOutOfTest.push(from);
|
|
continue;
|
|
}
|
|
// test → test: compare the original (base) against the relocated (head) file so
|
|
// a clean relocation passes but a rename that drops asserts/adds tautologies fires.
|
|
const baseSrc = git(["show", `${base}:${from}`]);
|
|
const headSrc = fs.existsSync(to) ? fs.readFileSync(to, "utf8") : "";
|
|
renamePerFile.push({
|
|
file: to,
|
|
baseAsserts: countAssertions(baseSrc),
|
|
headAsserts: countAssertions(headSrc),
|
|
baseTaut: countTautologies(baseSrc),
|
|
headTaut: countTautologies(headSrc),
|
|
baseSkips: countSkips(baseSrc),
|
|
headSkips: countSkips(headSrc),
|
|
baseExtTaut: countExtendedTautologies(baseSrc),
|
|
headExtTaut: countExtendedTautologies(headSrc),
|
|
});
|
|
}
|
|
|
|
// Arquivos de teste modificados (subcheck original + skips + extTaut)
|
|
const changed = git(["diff", "--name-only", "--diff-filter=M", `${base}...HEAD`])
|
|
.split("\n")
|
|
.map((s) => s.trim())
|
|
.filter((f) => TEST_RE.test(f) && fs.existsSync(f));
|
|
|
|
// (gap 6) A release PR is not a reviewable unit, and this is where that stops being free.
|
|
// Releases squash-merge into `main`, so a release PR's merge-base is the PREVIOUS cycle's
|
|
// fork point and the diff spans the whole cycle. In the v3.8.49 run that was ~1277 changed
|
|
// test files, each costing a `git show base:file` process plus a full regex pass — the check
|
|
// ran twice without finishing, >30 min pegged on one core, and the release waited on it.
|
|
//
|
|
// Every one of those files was already gated by this same check on its own PR during the
|
|
// cycle. Re-analyzing the aggregate buys nothing and blocks the release, so above the
|
|
// threshold the per-file diff subchecks are skipped — LOUDLY, naming the count, because a
|
|
// silent skip is how a gate becomes indistinguishable from a passing one (that is gap 12,
|
|
// and it cost two production bugs this cycle).
|
|
//
|
|
// The floor is untouched: scanBareTautologies() above already ran unconditionally over all
|
|
// tracked test files (3977 files, ~1 s), so nothing here lowers absolute coverage.
|
|
const maxChangedTests = Number(process.env.TEST_MASKING_MAX_CHANGED_TESTS || 300);
|
|
if (shouldSkipDiffSubchecks(changed.length + renamePerFile.length, maxChangedTests)) {
|
|
console.log(
|
|
`[test-masking] ${changed.length} teste(s) modificado(s) + ${renamePerFile.length} ` +
|
|
`renomeado(s) excede o teto de ${maxChangedTests} — pulando os subchecks de diff.\n` +
|
|
` Um diff desse tamanho é um PR de release (base = main, merge-base = fork do ciclo ` +
|
|
`anterior por causa do squash), não uma unidade revisável.\n` +
|
|
` Cada um desses arquivos já passou por este mesmo gate no PR de origem.\n` +
|
|
` O scan absoluto de tautologias rodou sobre TODOS os testes rastreados e está OK.\n` +
|
|
` Para forçar a análise completa: TEST_MASKING_MAX_CHANGED_TESTS=999999`
|
|
);
|
|
return;
|
|
}
|
|
|
|
const perFile = [...renamePerFile];
|
|
for (const file of changed) {
|
|
const baseSrc = git(["show", `${base}:${file}`]);
|
|
const headSrc = fs.readFileSync(file, "utf8");
|
|
perFile.push({
|
|
file,
|
|
baseAsserts: countAssertions(baseSrc),
|
|
headAsserts: countAssertions(headSrc),
|
|
baseTaut: countTautologies(baseSrc),
|
|
headTaut: countTautologies(headSrc),
|
|
baseSkips: countSkips(baseSrc),
|
|
headSkips: countSkips(headSrc),
|
|
baseExtTaut: countExtendedTautologies(baseSrc),
|
|
headExtTaut: countExtendedTautologies(headSrc),
|
|
});
|
|
}
|
|
|
|
// Per-file allowlist for verified-legitimate net-assert reductions (refactor/field-removal).
|
|
// Only exempts the reduction signal; tautology/skip/deletion signals still fire.
|
|
let assertReductionAllowlist = new Set();
|
|
let deletionAllowlist = {};
|
|
let reimplementedAllowlist = new Set();
|
|
try {
|
|
const raw = JSON.parse(fs.readFileSync("config/quality/test-masking-allowlist.json", "utf8"));
|
|
assertReductionAllowlist = new Set(Object.keys(raw).filter((k) => !k.startsWith("_")));
|
|
deletionAllowlist = raw._deletedWithReplacement || {};
|
|
reimplementedAllowlist = new Set(raw._reimplementedConditions || []);
|
|
} catch {
|
|
// no allowlist file — treat as empty
|
|
}
|
|
|
|
// (6348 subcheck 4, REPORT-ONLY) Tests that inline-reimplement a prod condition
|
|
// instead of importing the symbol that owns it. Prod files changed in this PR
|
|
// (added/copied/modified TS sources) are the reference corpus; each changed test
|
|
// file is scanned against them. Warns only — it never fails the gate for now.
|
|
const prodChanged = git(["diff", "--name-only", "--diff-filter=ACM", `${base}...HEAD`])
|
|
.split("\n")
|
|
.map((s) => s.trim())
|
|
.filter((f) => PROD_SRC_RE.test(f) && !TEST_RE.test(f) && fs.existsSync(f));
|
|
const prodSources = prodChanged.map((f) => {
|
|
try {
|
|
return fs.readFileSync(f, "utf8");
|
|
} catch {
|
|
return "";
|
|
}
|
|
});
|
|
const changedTests = git(["diff", "--name-only", "--diff-filter=ACM", `${base}...HEAD`])
|
|
.split("\n")
|
|
.map((s) => s.trim())
|
|
.filter((f) => TEST_RE.test(f) && fs.existsSync(f));
|
|
const reimplementedFlags = [];
|
|
if (prodSources.length) {
|
|
for (const tf of changedTests) {
|
|
if (reimplementedAllowlist.has(tf)) continue;
|
|
const src = fs.readFileSync(tf, "utf8");
|
|
for (const hit of findReimplementedConditions(prodSources, src, extractImports(src))) {
|
|
reimplementedFlags.push(
|
|
`${tf}: re-implementa a condição \`${hit.condition}\`` +
|
|
(hit.owner ? ` (dona: ${hit.owner})` : "") +
|
|
" — asserte através do import real em vez de copiar a condição"
|
|
);
|
|
}
|
|
}
|
|
}
|
|
if (reimplementedFlags.length) {
|
|
console.warn(
|
|
`[test-masking] (report-only) ${reimplementedFlags.length} teste(s) re-implementam ` +
|
|
`condição de produção em vez de importar o símbolo dono (classe #6216):\n` +
|
|
reimplementedFlags.map((f) => " ⚠ " + f).join("\n") +
|
|
`\n → importe o símbolo/função dono e asserte através dele (evita contrato duplicado ` +
|
|
`que diverge silenciosamente). Report-only por enquanto — não falha o gate.`
|
|
);
|
|
}
|
|
|
|
const deletedFlags = evaluateDeletedFiles(
|
|
[...deletedTests, ...relocatedOutOfTest],
|
|
deletionAllowlist
|
|
);
|
|
const maskingFlags = evaluateMasking(perFile, assertReductionAllowlist);
|
|
const allFlags = [...deletedFlags, ...maskingFlags];
|
|
|
|
if (allFlags.length) {
|
|
console.error(
|
|
`[test-masking] ${allFlags.length} sinal(is) de enfraquecimento de teste:\n` +
|
|
allFlags.map((f) => " ✗ " + f).join("\n") +
|
|
`\n → se a redução é legítima (refator/consolidação), explique no PR; senão, restaure os asserts.`
|
|
);
|
|
process.exit(1);
|
|
}
|
|
console.log(
|
|
`[test-masking] OK — ${changed.length} modificado(s), ${renames.length} renomeado(s) (relocação), ` +
|
|
`${deletedTests.length} deletado(s) — sem enfraquecimento`
|
|
);
|
|
}
|
|
|
|
if (import.meta.url === pathToFileURL(process.argv[1] || "").href) main();
|