mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-20 14:12:59 +03:00
chore(quality): close v3.8.28 cycle gate drift (re-baseline + nightly-mutation scope) (#4135)
* chore(quality): re-baseline v3.8.28 cycle drift (eslint/openapi/i18n)
The post-release push->main Quality Ratchet (run 27725117464) failed on 3
metrics that drifted during the v3.8.28 cycle from legitimate feature merges:
eslintWarnings 3769->3779, openapiCoverage.pct 38.3->37.6, i18nUiCoverage.pct
80.1->79.1. Reproduced locally on release/v3.8.29 (9f14c1294) — identical to CI.
None is hand-cleanable without gaming or unavailable infra:
- eslint +10: no-explicit-any is warn-level and allowed in tests (the drift is
test `any`), plus 4 react-hooks/exhaustive-deps in RequestLoggerV2.tsx (a
refresh-sensitive component — touching hook deps without UI tests is risky).
- openapi -0.7: the drop is from new INTERNAL routes (/api/tools/agent-bridge/*,
LOCAL_ONLY) — documenting them in the public spec would game the metric.
- i18n -1.0: 37/41 locales need ~3000 translations via `npm run i18n:run`, which
requires OMNIROUTE_TRANSLATION_API_KEY (not available locally).
Re-baselined to the real measured values per the documented v3.8.26 precedent
(_eslint_rebaseline_2026_06_16_v3826_forward_merge). Tighten at cycle end:
eslint/openapi via --require-tighten; i18n via i18n:run with creds.
* ci(mutation): scope nightly to 6 modules to fit the 180min budget
The full 8-module Stryker run timed out at the 180min nightly cap (run
27705123780: 16:47:33 -> 19:47:48 = exactly 180min; the prior 120min scheduled
run also timed out). #4078's DATA_DIR isolation made concurrency=4 safe but not
sufficient: the tap-runner re-spawns a node process per test file per mutant, so
spawn cost dominates and the two god-files chatCore.ts (5874 LOC) + combo.ts
(5282 LOC) — ~2/3 of the ~15k mutants — do not fit.
Removed both god-files from stryker.conf.json `mutate`. The 6 smaller modules fit
the budget and produce real mutation scores now; the god-files regain coverage
after the Onda 3 split into smaller units. Updated the nightly job name/comments
(8->6 modules). Removing entries from `mutate` does not affect the dry-run
baseline (it runs tap.testFiles, unchanged), so the all-green baseline is preserved.
This commit is contained in:
committed by
GitHub
parent
9e5c00d716
commit
34bde609ae
13
.github/workflows/nightly-mutation.yml
vendored
13
.github/workflows/nightly-mutation.yml
vendored
@@ -9,17 +9,20 @@ permissions:
|
||||
|
||||
jobs:
|
||||
stryker:
|
||||
name: Stryker mutation testing (8 critical modules — advisory)
|
||||
name: Stryker mutation testing (6 critical modules — advisory)
|
||||
runs-on: ubuntu-latest
|
||||
# Mutation testing is expensive (~15k mutants across 8 modules). It runs only
|
||||
# Mutation testing is expensive (~5k mutants across 6 modules). It runs only
|
||||
# on the nightly schedule / manual dispatch, never on PRs. The score is NOT
|
||||
# yet enforced as a ratchet (wired in a later INT phase) — for now the job
|
||||
# just produces the HTML/JSON report and uploads it as an artifact.
|
||||
#
|
||||
# Runs at stryker concurrency=4 (was 1) now that ./tests/_setup/isolateDataDir.ts
|
||||
# gives each test process its own DATA_DIR — at concurrency=1 the full phase did
|
||||
# not finish inside the timeout (the cancelled 2026-06-16/17 runs). 180 min leaves
|
||||
# headroom for the first cold run before the incremental cache is seeded.
|
||||
# gives each test process its own DATA_DIR. The full 8-module set still TIMED OUT
|
||||
# at the 180min cap (run 27705123780 = exactly 180min), because the two god-files
|
||||
# chatCore.ts/combo.ts dominate ~2/3 of the mutants; they were removed from
|
||||
# stryker.conf.json `mutate` until the Onda 3 split (see _mutate_godfiles_excluded_comment
|
||||
# there). The remaining 6 modules fit the budget. 180min leaves cold-run headroom
|
||||
# before the incremental cache is seeded.
|
||||
timeout-minutes: 180
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
"_comment": "Catraca de qualidade. 'down' = nao pode aumentar; 'up' = nao pode cair. Atualize via 'npm run quality:ratchet -- --update' (somente quando melhora). Cada valor e um numero REAL medido, nunca um chute. Cobertura entra na Fase 4 a partir de um run de cobertura mergeada no CI.",
|
||||
"metrics": {
|
||||
"eslintWarnings": {
|
||||
"value": 3769,
|
||||
"value": 3779,
|
||||
"direction": "down"
|
||||
},
|
||||
"eslintErrors": {
|
||||
@@ -80,12 +80,12 @@
|
||||
"tightenSlack": 10
|
||||
},
|
||||
"openapiCoverage.pct": {
|
||||
"value": 38.3,
|
||||
"value": 37.6,
|
||||
"direction": "up",
|
||||
"eps": 0.5
|
||||
},
|
||||
"i18nUiCoverage.pct": {
|
||||
"value": 80.1,
|
||||
"value": 79.1,
|
||||
"direction": "up",
|
||||
"eps": 0.5
|
||||
},
|
||||
@@ -157,5 +157,6 @@
|
||||
"_require_tighten_advisory_2026_06_15": "Fase 6A.5: o gate --require-tighten (falha quando uma metrica melhora sem o baseline ser apertado no mesmo PR) foi finalmente WIRADO no CI — porem como STEP ADVISORY (continue-on-error) no job quality-gate, NAO bloqueante. Motivo: (a) o proprio plano sequencia o aperto para 'o fim do ciclo', e a v3.8.26 acabou de abrir; (b) liga-lo bloqueante durante merges ativos travaria qualquer melhoria de metrica ate re-baseline. Para promover a BLOQUEANTE no fim do ciclo: remover o 'continue-on-error: true' do step 'Require-tighten' no ci.yml. tightenSlack nas metricas coverage.* impede falso-disparo pelo gap anti-flake.",
|
||||
"_require_tighten_flip_blocking_2026_06_16_v3827": "Fim do ciclo v3.8.27: o step 'Require-tighten' do job quality-gate (ci.yml) foi PROMOVIDO de ADVISORY para BLOQUEANTE (removido o 'continue-on-error: true', renomeado para 'Require-tighten (blocking)'). Verificado LIMPO (exit 0, '24 metricas, 0 melhoraram') no tip de release/v3.8.27 ANTES do flip: as metricas deterministicas medem == baseline (eslintWarnings 3769, eslintErrors 0) e as coverage.* foram avaliadas contra a cobertura mergeada == baseline (replicando o estado do CI, onde o job baixa o artifact coverage-report antes do collect). tightenSlack nas coverage.* (5 global / 10 por-modulo) absorve a variancia local-vs-CI e impede falso-disparo. Nenhuma metrica melhorou-sem-apertar, logo o flip nao reda o CI verde atual.",
|
||||
"_osv_flip_blocking_2026_06_16_v3827": "Fim do ciclo v3.8.27: vulnCount (value 10, direction down, dedicatedGate) PROMOVIDO de ADVISORY para RATCHET BLOQUEANTE. check-vuln-ratchet.mjs ganhou um modo --ratchet (espelhando check-secrets/check-bundle-size/check-workflows da Etapa 2): le metrics.vulnCount.value daqui, compara a contagem MEDIDA pelo osv-scanner e sai 1 SOMENTE numa regressao real (medida > baseline). Sem --ratchet permanece advisory (exit 0). QUALQUER SKIP gracioso (osv-scanner ausente do PATH, osv.dev/rede inacessivel, timeout, JSON invalido) sai 0 MESMO com --ratchet — uma falha de MEDICAO nunca bloqueia, so uma regressao MEDIDA bloqueia. No ci.yml (job quality-extended) o step 'Vulnerability ratchet' passou a rodar 'npm run check:vuln-ratchet -- --ratchet' sem continue-on-error. Verificado no tip de release/v3.8.27: osv mede vulnCount=10 == baseline 10 (exit 0); forcar baseline=9 da exit 1 (10>9); PATH vazio da SKIP binary-absent exit 0. NB de VARIANCIA DE CVE: um CVE recem-divulgado numa dep ja presente pode redar o gate sem mudanca de dependencia — comportamento esperado, remedio = bumpar a dep ou re-baseline vulnCount com justificativa+issue. Ver docs/security/SUPPLY_CHAIN.md -> 'Variancia de CVE'.",
|
||||
"_trivy_flip_blocking_2026_06_16_v3827": "Fim do ciclo v3.8.27: Trivy (scan de CVE da imagem Docker em docker-publish.yml) promovido para BLOQUEANTE em CRITICAL. Abordagem de DOIS PASSOS: o passo SARIF existente (severity HIGH,CRITICAL / exit-code 0 / upload SARIF) fica INTACTO para visibilidade na aba Security; um novo passo 'Trivy CRITICAL gate (blocking)' (severity CRITICAL / ignore-unfixed:true / exit-code 1) falha o release num CVE CRITICO FIXAVEL. ignore-unfixed evita travar por CVE de base-image sem patch upstream (reduz falso-bloqueio). Mesma variancia-de-CVE do osv: um novo CRITICAL fixavel divulgado pode redar; remedio = rebuild sobre base patcheada, bumpar dep, ou .trivyignore com justificativa+issue. Ver docs/security/SUPPLY_CHAIN.md. vulnCount permanece 10 (intocado neste flip — so a postura advisory->bloqueante mudou)."
|
||||
"_trivy_flip_blocking_2026_06_16_v3827": "Fim do ciclo v3.8.27: Trivy (scan de CVE da imagem Docker em docker-publish.yml) promovido para BLOQUEANTE em CRITICAL. Abordagem de DOIS PASSOS: o passo SARIF existente (severity HIGH,CRITICAL / exit-code 0 / upload SARIF) fica INTACTO para visibilidade na aba Security; um novo passo 'Trivy CRITICAL gate (blocking)' (severity CRITICAL / ignore-unfixed:true / exit-code 1) falha o release num CVE CRITICO FIXAVEL. ignore-unfixed evita travar por CVE de base-image sem patch upstream (reduz falso-bloqueio). Mesma variancia-de-CVE do osv: um novo CRITICAL fixavel divulgado pode redar; remedio = rebuild sobre base patcheada, bumpar dep, ou .trivyignore com justificativa+issue. Ver docs/security/SUPPLY_CHAIN.md. vulnCount permanece 10 (intocado neste flip — so a postura advisory->bloqueante mudou).",
|
||||
"_rebaseline_2026_06_18_v3828_cycle_close": "Fim do ciclo v3.8.28 (RELEASED; ciclo v3.8.29 aberto): 3 metricas re-baselineadas para o valor REAL medido no push->main pos-release (run 27725117464, step 'Ratchet check') — eslintWarnings 3769->3779, openapiCoverage.pct 38.3->37.6, i18nUiCoverage.pct 80.1->79.1. Reproduzido localmente em release/v3.8.29 (9f14c1294): identico ao CI. Drift de fim-de-ciclo de features legitimas, NAO hand-cleanable: (a) eslint +10 = 'any' PERMITIDO (warn) em testes do ciclo + 4 react-hooks/exhaustive-deps em RequestLoggerV2.tsx (componente com bugs sutis de refresh #4103/#3972, arriscado mexer em deps de hook sem teste de UI); (b) openapi -0.7 = drop por rotas NOVAS INTERNAS (/api/tools/agent-bridge/* LOCAL_ONLY, spawnam MITM/DNS) — documenta-las no spec PUBLICO seria gaming; (c) i18n -1.0 = 37/41 locales em 79.1% (1741 chaves faltando cada, ~3000 traducoes via 'npm run i18n:run' que exige creds OMNIROUTE_TRANSLATION_API_KEY indisponiveis localmente). Mesmo precedente do _eslint_rebaseline_2026_06_16_v3826_forward_merge. Apertar no fim do ciclo: eslint/openapi via --require-tighten; i18n via i18n:run com creds. Autorizado pelo operador (decisao explicita)."
|
||||
}
|
||||
|
||||
@@ -185,9 +185,18 @@
|
||||
"{{testFile}}"
|
||||
]
|
||||
},
|
||||
"_mutate_godfiles_excluded_comment": [
|
||||
"2026-06-18 (Onda 2 budget): chatCore.ts (5874 LOC) + combo.ts (5282 LOC) — the two",
|
||||
"god-files — were REMOVED from `mutate`. They dominate ~2/3 of the ~15k mutants, and the",
|
||||
"full 8-module run TIMED OUT at the 180min nightly cap (run 27705123780, concurrency=4 +",
|
||||
"DATA_DIR isolation: started 16:47:33 -> killed 19:47:48 = exactly 180min; the prior 120min",
|
||||
"scheduled run also timed out). #4078 made concurrency safe but the tap-runner re-spawns a",
|
||||
"node process per test file PER MUTANT, so spawn cost dominates and 15k mutants does not fit.",
|
||||
"The 6 smaller modules below fit the budget and produce REAL mutation scores now; the two",
|
||||
"god-files regain mutation coverage after the Onda 3 split into smaller units (re-add them",
|
||||
"here — ideally as the split sub-modules). See project memory: Quality Gate v2 / Fase 9."
|
||||
],
|
||||
"mutate": [
|
||||
"open-sse/handlers/chatCore.ts",
|
||||
"open-sse/services/combo.ts",
|
||||
"open-sse/services/accountFallback.ts",
|
||||
"src/sse/services/auth.ts",
|
||||
"src/server/authz/routeGuard.ts",
|
||||
|
||||
Reference in New Issue
Block a user