fix(ci): stabilize nightly-mutation — guard tap.testFiles drift + anti-flake eps (#5682)

Root cause (NOT a timeout): the nightly-mutation run fails on cold-cache nights
because the blocking mutation-ratchet job measures modules below baseline, while
warm-cache nights pass — the verdict tracked GitHub Actions cache state, not code
quality. Proven via a local Stryker probe on headers.ts: covering unit tests
(no-memory-header, strip-reasoning) had drifted OUT of stryker.conf.json
tap.testFiles, so their mutants went covered-but-unkilled = Survived on a cold
full run (COVERED score 61.73 vs 94.29 baseline); adding them restores the kills.

- Add scripts/check/check-mutation-test-coverage.mjs: guards that every UNIT test
  importing a Stryker-mutated module is listed in tap.testFiles. Advisory by
  default, --strict in CI (wired in quality.yml fast-gates). Prevents recurrence.
- Add the 38 drifted covering unit tests to stryker.conf.json tap.testFiles
  (138 -> 176). Monotonically safe: more covering tests only raise/hold the score.
- Add MUTATION_RATCHET_EPS (1.0pt) anti-flake tolerance to check-mutation-ratchet
  so sub-point tap-runner jitter no longer false-fails the gate. Lowers no baseline.
- Tests: check-mutation-test-coverage (3) + eps cases in check-mutation-ratchet.

Residual: a clean post-merge nightly confirms scores return to/above baseline;
any marginal residual gets a baseline re-seed (operator).
This commit is contained in:
Diego Rodrigues de Sa e Souza
2026-06-30 13:51:47 -03:00
committed by GitHub
parent b5f8940399
commit 39d1fc5c13
7 changed files with 265 additions and 4 deletions

View File

@@ -52,6 +52,10 @@ jobs:
- run: npm run check:route-guard-membership
- run: npm run check:test-discovery
- run: npm run check:test-runner-api
# Guards tap.testFiles drift: a covering unit test absent from stryker.conf.json
# tap.testFiles makes its module's mutants survive on a cold nightly-mutation run,
# false-failing the blocking mutationScore ratchet. See check-mutation-test-coverage.mjs.
- run: npm run check:mutation-test-coverage
- run: npm run check:any-budget:t11
# Build-scope guard: fails if worktrees/cruft leak into the tsconfig include
# scope (would OOM `next build`). Instant. See incident 2026-06-25 / #5031.