feat(quality): seed per-module mutationScore floors + blocking aggregation ratchet (T3) (#4305)

First full mutation measurement landed (run 27823984918, the split nightly from #4272):
31 modules now have a COVERED mutation score. T3 turns that into an enforced gate.

Seed: 31 `mutationScore.<path>` floors in quality-baseline.json at ~2pt below the measured
score (absorbs run-to-run variance), direction:up, dedicatedGate:true. dedicatedGate means
the generic check-quality-ratchet SKIPS them (check-quality-ratchet.mjs:62) — they are
enforced only by check-mutation-ratchet.mjs. Range: memorySkillsInjection 13.49 (weakest)
to headers 94.29 (strongest); the security/critical floors: auth 52.57, accountFallback
68.38, routeGuard 76.08, circuitBreaker 56.94, error 43.83, publicCreds 59.76.

Gate: a new `mutation-ratchet` job in nightly-mutation.yml runs AFTER all batches
(needs: stryker, if: always()), downloads every mutation report, and ratchets the MERGED
per-module scores with `check-mutation-ratchet --ratchet` (blocking). It must aggregate
because the split batches each emit a PARTIAL view of a file (auth.ts in a1+a2,
accountFallback in b1+b2) — a per-batch ratchet would compare half a file against the
whole-file floor. check-mutation-ratchet unions same-file mutants across reports (#4272).
A module dropping below its floor fails the run; missing reports (upload flake) are skipped.

Verified: ratchet exits 0 on the seeded measurements, exits 1 on a synthetic regression
(auth 33.33 < 52.57), exits 0 advisory without --ratchet. Baseline change is additive
(31 floors + one comment; existing keys untouched). check-mutation-ratchet tests 8/8.
This commit is contained in:
Diego Rodrigues de Sa e Souza
2026-06-19 18:28:06 -03:00
committed by GitHub
parent 23455fdb0a
commit bbc9d1e1c5
2 changed files with 186 additions and 1 deletions

View File

@@ -132,3 +132,32 @@ jobs:
path: reports/mutation/
if-no-files-found: warn
retention-days: 14
# Aggregation gate (T3): each split batch emits a PARTIAL view of a mutated file
# (auth.ts lives in a1+a2, accountFallback in b1+b2), so a PER-BATCH ratchet would
# only ever see half a file vs the whole-file baseline. This job runs AFTER every
# batch, downloads all reports, and ratchets the MERGED per-module scores
# (check-mutation-ratchet UNIONS same-file mutants across reports) against the
# dedicatedGate `mutationScore.*` floors in quality-baseline.json (seeded ~2pt below
# the first full measurement). Blocking: a module dropping below its floor fails the
# run. Missing reports (e.g. an artifact-upload flake) are skipped, never failed.
mutation-ratchet:
name: Mutation score ratchet (blocking)
needs: stryker
if: always()
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v6
with:
persist-credentials: false
- uses: actions/setup-node@v6
with:
node-version: "24"
- name: Download all mutation reports
uses: actions/download-artifact@v8
with:
pattern: mutation-report-*
path: reports/all
- name: Ratchet merged per-module mutation scores
run: node scripts/check/check-mutation-ratchet.mjs reports/all/*/mutation.json --ratchet