mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-07-31 12:22:14 +03:00
* chore(release): open v3.8.19 development cycle * chore(release): sync electron lockfile to 3.8.19 * feat(quality): quality-gate ratchet + anti-hallucination/rule-enforcement guardrails (Phases 0-6) (#3471) * feat(quality): generic ratchet comparator (multi-metric, regression-only) * chore(ci): Fase 0 quality-gate fixes — reconcile coverage gate (40->60), tier npm audit, wire orphaned contract gates, re-enable cheap husky pre-commit * feat(quality): ratchet engine (collector + frozen baseline + CI job) and provider-consistency gate - collect-metrics.mjs: emits quality-metrics.json (ESLint warnings + coverage when present) - quality-baseline.json: frozen baseline (eslintWarnings=3482, regression-only) - ci.yml: quality-gate job (ratchet + step summary + artifact) and check:provider-consistency in lint job - check-provider-consistency.ts: every REGISTRY id must be a canonical provider (found krutrim half-registered → allowlisted as known pre-existing, blocks any NEW orphan) - TDD: 9 tests (5 ratchet + 4 provider-consistency) * feat(quality): Fase 2 anti-hallucination gates — fetch-targets, openapi-routes, deps allowlist - check-fetch-targets: every dashboard fetch(/api/...) resolves to a real route.ts; found 7 pre-existing dashboard->route mismatches frozen as KNOWN_MISSING for triage - check-openapi-routes: every openapi.yaml path resolves to a real route; found 1 stale spec entry (agent-bridge agents/{id}/state) frozen as KNOWN_STALE_SPEC - check-deps: anti-slopsquatting allowlist (105 deps); new deps need explicit human-reviewed entry - all wired into CI lint/docs jobs; TDD +12 tests (21 total across 5 gates) * docs(quality): add quality-gates report + implementation plan to repo root * feat(quality): Fase 3a — file-size ratchet (freeze 91 files >800 LOC, cap 800 for new) - check-file-size.mjs: frozen files can only shrink; new files must be <= cap (kills the next 12k-line god-component) - file-size-baseline.json: 91 files frozen at current LOC (largest 12883) - wired into CI lint job; TDD 5 tests; --update ratchets the baseline down on shrink * feat(quality): Fase 3b — duplication ratchet (jscpd@4, baseline 5.72%) - check-duplication.mjs: runs jscpd@4 (pinned; v5 is an incompatible Rust rewrite) over src+open-sse, fails if duplication % rises vs frozen baseline (5.72%, measured: 1358 clones / 22967 dup lines). Targets the executor copy-paste (48/50 override execute() wholesale) - wired into the parallel quality-gate CI job (off the lint critical path); TDD 4 tests; --update ratchets down - snapshot now complete: coverage ~82.6%, eslint 3482 (98.5% no-explicit-any), duplication 5.72%, 91 files >800 LOC * feat(quality): Fase 4a — anti test-masking gate - check-test-masking.mjs: for each MODIFIED test file in a PR, flags net assert removal + new assert.ok(true) tautologies (base...HEAD diff). Directly enforces CLAUDE.md 'never weaken asserts to go green' - wired into pr-test-policy CI job (reuses base fetch); no-op outside PR; TDD 5 tests * feat(quality): Fase 4b — coverage ratchet (conservative floors, CI consumes merged coverage) - quality-baseline.json: coverage.{statements,lines,functions,branches} floors (80/80/82/73, real ~82.58/82.58/84.23/75.22 with margin; tighten via --update after a green main run) - check-quality-ratchet.mjs: --allow-missing (local quality:gate skips coverage.* without a coverage run; CI runs strict) - ci.yml quality-gate job: needs test-coverage + downloads merged coverage-report so the ratchet enforces 'coverage cannot drop' - TDD +1 test (6 total) * feat(quality): Fase 6 — 8 new gates (Rule #11/#12, migrations, known-symbols, route-guard, complexity, docs-symbols, db-rules) Deterministic gates, each freezing pre-existing violations in a documented allowlist (ratchet) so they pass now and block only NEW regressions: - check-error-helper (Rule #12): 7 executors/handlers forwarding raw err.message frozen - check-public-creds (Rule #11): 5 literal client_ids (Claude/Codex/Qwen/Kimi/Copilot) frozen - check-migration-numbering: gaps 026/055 + dup 041 frozen (prevents the git-rm-deleted-migration incident) - check-known-symbols: 93 executors conformance + 15 combo strategies + 18 translator pairs - check-route-guard-membership (#15/#17): all 25 spawn-capable routes verified local-only (0 gaps) - check-complexity: cyclomatic>15 / fn-length>80 ratchet (baseline 1739) - check-docs-symbols: 30 stale doc /api refs frozen (docs hallucination) - check-db-rules (#2/#5): 25 unexported db modules + 15 raw-SQL routes frozen Wired into CI (lint / docs-sync-strict / quality-gate jobs). 115 TDD tests, all green. ESLint ratchet held at 3482. * docs(quality): Phase 7 plan (security/dead-code/mutation/community tooling) — GATED to 2026-06-16 Stored, not active. 7 suggested gates + all discussed OSS/Community tools (SonarQube Community + osv-scanner + CodeQL + knip + sonarjs + type-coverage + lockfile-lint + Stryker + size-limit + axe-core + semcheck + agent-lsp + Qlty). Activation gate: do not start before 2026-06-16 (use Phases 0-6 in production for 1 week, validate in practice, then evolve). * docs(quality): Phase 6A critical-audit plan + Phase 7 additions — gated to 2026-06-16 (#3530) PLANO-QUALITY-GATES-FASE6A.md (12-task audit of Phases 0-6: orphan tests, stale-allowlist enforcement, scope gaps) + Phase 7 additions (gitleaks, actionlint+zizmor, license compliance). Both stored, activation gated to 2026-06-16. Tasks 6A.1/6A.2 were fast-tracked separately (#3536). * feat(quality): 6A.1+6A.2 — test-discovery gate, 135 orphan tests re-wired, 2 production bug fixes, vitest in CI (#3536) check-test-discovery gate (TDD; 195 orphans found, 135 re-wired into the node runner, 60 frozen+annotated). Triage fixed 2 real production bugs: missing BYPASS_PREFIX_NOT_ALLOWED zod refine (spawn-capable prefixes accepted into the bypass list, Hard Rules #15/#17) and resetDbInstance not firing stateReset resetters (stale schema memo → 503 instead of 403; also hit backup-restore). New test-vitest CI job: test:vitest blocking (146/146), test:vitest:ui informational (14 pre-existing fails, triage 2026-06-16). * chore: ignore generated yt-downloader artifact files Add dated yt-downloader output files to .gitignore to prevent local automation artifacts from being accidentally committed. * chore(quality): green-light the quality-gate — conscious file-size + eslintWarnings re-baselines (#3538) file-size: 9 files frozen at current sizes (v3.8.18-era growth + core.ts +7 from #3536 fix). eslintWarnings 3482→3501: the published v3.8.18 tag already measures 3501 (delta predates the quality-gate job); v3.8.19 cycle is neutral. Reduction + --require-tighten = Phase 6A (2026-06-16). * fix(check): exclude internal planning docs (docs/superpowers/) from the docs-symbols gate docs/superpowers/plans/*.md are historical implementation-plan snapshots that may cite planned/abandoned routes — not claims about the current code. Three such refs entered during the v3.8.18 cycle, before this gate was on the pipeline, and would have blocked the v3.8.19 release merge. * chore(release): v3.8.19 — 2026-06-09 CHANGELOG section for the quality-infrastructure release (7 commits, 1:1 coverage), [3.8.18] label corrected to its real release date, local prompt artifacts ignored. * test: hermetic auth context for 2 re-wired suites + real headroom on the breaker reset-timeout flake CI shards exposed what the dev DATA_DIR was masking locally: detect.test.ts and managementCliToken.test.ts asserted 401/403/reject outcomes that only exist when login protection is configured — on a fresh CI DB isAuthRequired() is false and the policy anonymous-allows. Both now create an isolated DATA_DIR with requireLogin+password (the established pattern). observability-fase04: the breaker reset-timeout test ran with a 5ms margin (resetTimeout 10 / sleep 15) — lazy HALF_OPEN refresh under shard contention flipped the first OPEN assert. Now 250/300ms. * test: align bypass-prefix schema test to the restored layer-1 contract + real waitFor headroom appearance-widget-settings-schema asserted that /api/cli-tools/runtime/ was ACCEPTED into the bypass list — written against the buggy schema (missing BYPASS_PREFIX_NOT_ALLOWED refine, restored in #3536) and consecrating the bug the AC-8 orphan test guards against. Split into accept-safe + reject-spawn-capable cases. chatcore waitFor ceiling 1500→10000ms (green runs return immediately; observed 1580ms expiry on 2-core CI runners). * test(chatcore): fix structurally-broken pending-detail predicate (flatten before find) pendingRequests.details[connectionId] is Record<modelKey, PendingRequestDetail[]> — the upstream-timeout test's waitFor tested each ARRAY's .providerRequest (always undefined), so it could never resolve and expired (failed on 3 CI jobs; reproduced deterministically isolated, including at the published v3.8.18 tag). Flatten to the actual details + declare the call_log_pipeline_enabled dependency explicitly + waitFor ceiling with real CI headroom. * chore(quality): re-baseline coverage floors to the honest post-re-wire denominator + changelog coverage for the stabilization commits The 135 re-wired tests import modules that were never loaded before, so the c8 denominator grew: the old ~82.5% was inflated by never-imported modules being invisible. CI merged coverage now measures 78.4/78.4/83.84/75.73 — floors set ~2pt below (76.5/76.5; functions/branches floors already hold). Tightening via --require-tighten is Phase 6A work (2026-06-16).
247 lines
11 KiB
JavaScript
247 lines
11 KiB
JavaScript
#!/usr/bin/env node
|
|
// scripts/check/check-test-discovery.mjs
|
|
// Gate 6A.1 — test discovery: todo arquivo *.test.ts|tsx / *.spec.ts|tsx do repo deve
|
|
// ser COLETADO por pelo menos um runner que efetivamente RODA via npm script ou CI.
|
|
//
|
|
// WHY: a auditoria 2026-06-09 encontrou ≈135 testes em subdiretórios de tests/unit/
|
|
// que nenhum runner coleta (o glob `tests/unit/*.test.ts` é top-level-only), incluindo
|
|
// tests/unit/authz/routeGuard.test.ts (Hard Rules #15/#17) — cujos asserts JÁ FALHAM,
|
|
// apodrecidos sem ninguém ver. Teste que não roda é o falso verde definitivo: todo o
|
|
// investimento anti test-masking protege asserts que nem executam.
|
|
//
|
|
// Modelo: COLLECTORS declara explicitamente o glob de cada runner REAL + as fontes
|
|
// (package.json / ci.yml / vitest configs) onde o padrão deve aparecer textualmente
|
|
// (drift-check: mudou o glob na fonte sem atualizar aqui → o gate falha pedindo sync).
|
|
// "Coletado" = casado pelo glob de um runner executado por script npm ou job de CI.
|
|
// Includes de config que NENHUM script executa (ex.: vitest.config.ts sem filtro) NÃO
|
|
// contam — config morta não roda teste.
|
|
//
|
|
// Catraca: órfãos pré-existentes ficam congelados em test-discovery-baseline.json
|
|
// (dívida visível, decrescente). Órfão NOVO → fail. Entrada do baseline que deixou de
|
|
// ser órfã (religada/deletada) → fail pedindo remoção (stale-allowlist enforcement).
|
|
// --update regrava o baseline com o estado atual (use só para REMOVER religados;
|
|
// adições novas devem ser corrigidas, não congeladas — esse é o ponto do gate).
|
|
//
|
|
// Limitações documentadas (v1):
|
|
// - `exclude` de arquivo individual em vitest configs não é modelado (1 caso hoje:
|
|
// providerDiversity.test.ts — coletado pelo include, deliberadamente excluído).
|
|
// - @omniroute/* ficam fora do walk (têm CI próprio: opencode-*-ci.yml).
|
|
import fs from "node:fs";
|
|
import path from "node:path";
|
|
import { pathToFileURL } from "node:url";
|
|
|
|
const ROOT = process.cwd();
|
|
const BASELINE_PATH = path.resolve(
|
|
process.argv.includes("--baseline")
|
|
? process.argv[process.argv.indexOf("--baseline") + 1]
|
|
: path.join(ROOT, "test-discovery-baseline.json")
|
|
);
|
|
const UPDATE = process.argv.includes("--update");
|
|
|
|
// Raízes varridas em busca de arquivos de teste.
|
|
const WALK_ROOTS = ["tests", "src", "open-sse", "electron", "bin"];
|
|
const WALK_EXCLUDE = new Set(["node_modules", ".next", "dist", "coverage", ".git"]);
|
|
const TEST_FILE_RE = /\.(test|spec)\.(ts|tsx)$/;
|
|
|
|
// Runners REAIS e seus globs. `sources`: arquivos onde `anchor` (default: o próprio
|
|
// glob) deve aparecer textualmente — se o runner mudar, este gate exige o sync.
|
|
export const COLLECTORS = [
|
|
// Node native runner — test:unit / test:unit:fast / shards / test:coverage + CI (8 shards, node24, node26)
|
|
{ glob: "tests/unit/*.test.ts", sources: ["package.json", ".github/workflows/ci.yml"] },
|
|
// Node native runner — subdiretórios religados pela 6A.1c (2026-06-09). Braces
|
|
// explícitos para NÃO incluir tests/unit/autoCombo/** (testes vitest — importam
|
|
// "vitest" e explodem no node runner). Subdir novo: adicione aqui E nos scripts
|
|
// (o drift-check + o gate de órfãos forçam a manutenção em sincronia).
|
|
{
|
|
glob: "tests/unit/{api,auth,authz,build,cli,cli-helper,compression,cors,dashboard,db,db-adapters,docs,gamification,guardrails,lib,mcp,runtime,security,services,settings,shared,ui}/**/*.test.ts",
|
|
sources: ["package.json", ".github/workflows/ci.yml"],
|
|
},
|
|
// Node native runner — test:integration (top-level only; tests/integration/services/ NÃO roda)
|
|
{ glob: "tests/integration/*.test.ts", sources: ["package.json"] },
|
|
// Node native runner — test:system
|
|
{ glob: "tests/e2e/system-failover.test.ts", sources: ["package.json"] },
|
|
// vitest.mcp.config.ts — test:vitest
|
|
{ glob: "open-sse/mcp-server/__tests__/**/*.test.ts", sources: ["vitest.mcp.config.ts"] },
|
|
{ glob: "open-sse/services/autoCombo/__tests__/**/*.test.ts", sources: ["vitest.mcp.config.ts"] },
|
|
{ glob: "tests/unit/autoCombo/**/*.test.ts", sources: ["vitest.mcp.config.ts"] },
|
|
{ glob: "tests/unit/encryption.spec.ts", sources: ["vitest.mcp.config.ts"] },
|
|
{ glob: "src/shared/components/**/*.test.tsx", sources: ["vitest.mcp.config.ts"] },
|
|
{ glob: "src/shared/hooks/__tests__/**/*.test.tsx", sources: ["vitest.mcp.config.ts"] },
|
|
{ glob: "src/app/(dashboard)/**/__tests__/**/*.test.tsx", sources: ["vitest.mcp.config.ts"] },
|
|
// vitest.config.ts via test:vitest:ui (roda com path-filter `tests/unit/ui`, então o
|
|
// conjunto EFETIVO é a interseção do include `tests/unit/**/*.test.tsx` com o filtro)
|
|
{
|
|
glob: "tests/unit/ui/**/*.test.tsx",
|
|
sources: ["package.json", "vitest.config.ts"],
|
|
anchors: { "package.json": "tests/unit/ui", "vitest.config.ts": "tests/unit/**/*.test.tsx" },
|
|
},
|
|
// Playwright — test:e2e (o script passa tests/e2e/*.spec.ts; testMatch **/*.spec.ts)
|
|
{ glob: "tests/e2e/*.spec.ts", sources: ["package.json"] },
|
|
// Runners custom — test:ecosystem / test:protocols:e2e (spawnam vitest com o arquivo)
|
|
{ glob: "tests/e2e/ecosystem.test.ts", sources: ["scripts/dev/run-ecosystem-tests.mjs"] },
|
|
{ glob: "tests/e2e/protocol-clients.test.ts", sources: ["scripts/dev/run-protocol-clients-tests.mjs"] },
|
|
];
|
|
|
|
const escapeRe = (s) => s.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
|
|
|
|
/** Converte um glob em RegExp ancorada. Suporta `*`, `**` (com ou sem barra) e `{a,b}`. */
|
|
export function globToRegExp(glob) {
|
|
let re = "";
|
|
for (let i = 0; i < glob.length; i++) {
|
|
const c = glob[i];
|
|
if (c === "*") {
|
|
if (glob[i + 1] === "*") {
|
|
if (glob[i + 2] === "/") {
|
|
re += "(?:.*/)?"; // "**/" — zero ou mais diretórios
|
|
i += 2;
|
|
} else {
|
|
re += ".*"; // "**" solto
|
|
i += 1;
|
|
}
|
|
} else {
|
|
re += "[^/]*"; // "*" não atravessa "/"
|
|
}
|
|
} else if (c === "{") {
|
|
const end = glob.indexOf("}", i);
|
|
const alts = glob
|
|
.slice(i + 1, end)
|
|
.split(",")
|
|
.map(escapeRe);
|
|
re += "(?:" + alts.join("|") + ")";
|
|
i = end;
|
|
} else {
|
|
re += escapeRe(c);
|
|
}
|
|
}
|
|
return new RegExp("^" + re + "$");
|
|
}
|
|
|
|
/** Arquivos de teste não casados por NENHUM glob de collector (ordem preservada). */
|
|
export function findOrphans(files, globs) {
|
|
const regexes = globs.map(globToRegExp);
|
|
return files.filter((f) => !regexes.some((re) => re.test(f)));
|
|
}
|
|
|
|
/**
|
|
* Compara os órfãos atuais com o baseline congelado.
|
|
* - newOrphans: órfão atual fora do baseline → teste novo que NÃO RODA (fail).
|
|
* - stale: entrada do baseline que não é mais órfã (religada/deletada) → remova (fail).
|
|
*/
|
|
export function evaluateAgainstBaseline(orphans, baselineList) {
|
|
const baseSet = new Set(baselineList);
|
|
const orphanSet = new Set(orphans);
|
|
return {
|
|
newOrphans: orphans.filter((o) => !baseSet.has(o)),
|
|
stale: baselineList.filter((b) => !orphanSet.has(b)),
|
|
};
|
|
}
|
|
|
|
/**
|
|
* Drift-check: cada glob declarado (ou seu anchor por fonte) deve aparecer textualmente
|
|
* em TODAS as suas fontes. Retorna mensagens de drift.
|
|
*/
|
|
export function findCollectorDrift(collectors, contents) {
|
|
const drift = [];
|
|
for (const c of collectors) {
|
|
for (const source of c.sources) {
|
|
const anchor = c.anchors?.[source] ?? c.glob;
|
|
const body = contents[source];
|
|
if (body === undefined || !body.includes(anchor)) {
|
|
drift.push(
|
|
`glob "${c.glob}" (anchor "${anchor}") não encontrado em ${source} — o runner mudou? Sincronize COLLECTORS em check-test-discovery.mjs`
|
|
);
|
|
}
|
|
}
|
|
}
|
|
return drift;
|
|
}
|
|
|
|
function walk(dir, acc = []) {
|
|
if (!fs.existsSync(dir)) return acc;
|
|
for (const e of fs.readdirSync(dir, { withFileTypes: true })) {
|
|
if (WALK_EXCLUDE.has(e.name)) continue;
|
|
const p = path.join(dir, e.name);
|
|
if (e.isDirectory()) walk(p, acc);
|
|
else if (TEST_FILE_RE.test(e.name)) acc.push(p);
|
|
}
|
|
return acc;
|
|
}
|
|
|
|
function collectTestFiles() {
|
|
const out = [];
|
|
for (const root of WALK_ROOTS) {
|
|
for (const f of walk(path.join(ROOT, root))) {
|
|
out.push(path.relative(ROOT, f).replace(/\\/g, "/"));
|
|
}
|
|
}
|
|
return out.sort();
|
|
}
|
|
|
|
function main() {
|
|
// 1) drift dos collectors vs fontes reais
|
|
const contents = {};
|
|
for (const c of COLLECTORS) {
|
|
for (const s of c.sources) {
|
|
if (contents[s] === undefined) {
|
|
const p = path.join(ROOT, s);
|
|
contents[s] = fs.existsSync(p) ? fs.readFileSync(p, "utf8") : undefined;
|
|
}
|
|
}
|
|
}
|
|
const drift = findCollectorDrift(COLLECTORS, contents);
|
|
|
|
// 2) órfãos vs baseline
|
|
const files = collectTestFiles();
|
|
const orphans = findOrphans(
|
|
files,
|
|
COLLECTORS.map((c) => c.glob)
|
|
);
|
|
if (!fs.existsSync(BASELINE_PATH) && !UPDATE) {
|
|
console.error(
|
|
`[test-discovery] FAIL — ${path.basename(BASELINE_PATH)} ausente. Bootstrap:\n` +
|
|
` node scripts/check/check-test-discovery.mjs --update (gera o baseline com os órfãos atuais)`
|
|
);
|
|
process.exit(2);
|
|
}
|
|
const baseline = fs.existsSync(BASELINE_PATH)
|
|
? JSON.parse(fs.readFileSync(BASELINE_PATH, "utf8"))
|
|
: {
|
|
_comment:
|
|
"Catraca de test-discovery (check-test-discovery.mjs). Cada entrada e um arquivo de teste que NENHUM runner coleta (ele nunca roda) — divida congelada na auditoria 6A.1 (2026-06-09). So pode DIMINUIR: religue o teste (ajustando o glob do runner ou movendo o arquivo) e remova a entrada via --update. NAO adicione novos orfaos — corrija o runner.",
|
|
orphans: [],
|
|
};
|
|
const { newOrphans, stale } = evaluateAgainstBaseline(orphans, baseline.orphans || []);
|
|
|
|
if (UPDATE && drift.length === 0) {
|
|
baseline.orphans = orphans;
|
|
fs.writeFileSync(BASELINE_PATH, JSON.stringify(baseline, null, 2) + "\n");
|
|
console.log(
|
|
`[test-discovery] baseline regravado: ${orphans.length} órfão(s) (${stale.length} removido(s), ${newOrphans.length} adicionado(s) — adições devem ser corrigidas, não congeladas)`
|
|
);
|
|
return;
|
|
}
|
|
|
|
const problems = [];
|
|
for (const d of drift) problems.push(` ✗ [drift] ${d}`);
|
|
for (const o of newOrphans) {
|
|
problems.push(
|
|
` ✗ [órfão NOVO] ${o} — nenhum runner coleta este arquivo (ele NUNCA roda). Mova-o para um path coletado ou ajuste o runner.`
|
|
);
|
|
}
|
|
for (const s of stale) {
|
|
problems.push(
|
|
` ✗ [stale] ${s} — não é mais órfão (religado/removido). Remova do baseline: node scripts/check/check-test-discovery.mjs --update`
|
|
);
|
|
}
|
|
|
|
if (problems.length) {
|
|
console.error(`[test-discovery] ${problems.length} problema(s):\n` + problems.join("\n"));
|
|
process.exit(1);
|
|
}
|
|
console.log(
|
|
`[test-discovery] OK — ${files.length} arquivos de teste, ${COLLECTORS.length} collectors, ${(baseline.orphans || []).length} órfão(s) congelado(s) (dívida rastreada, só decresce)`
|
|
);
|
|
}
|
|
|
|
if (import.meta.url === pathToFileURL(process.argv[1] || "").href) main();
|