Files
OmniRoute/scripts/check/check-docs-symbols.mjs
Diego Rodrigues de Sa e Souza 5f2722bd6c feat(quality): quality-gate ratchet + anti-hallucination/rule-enforcement guardrails (Phases 0-6) (#3471)
* feat(quality): generic ratchet comparator (multi-metric, regression-only)

* chore(ci): Fase 0 quality-gate fixes — reconcile coverage gate (40->60), tier npm audit, wire orphaned contract gates, re-enable cheap husky pre-commit

* feat(quality): ratchet engine (collector + frozen baseline + CI job) and provider-consistency gate

- collect-metrics.mjs: emits quality-metrics.json (ESLint warnings + coverage when present)
- quality-baseline.json: frozen baseline (eslintWarnings=3482, regression-only)
- ci.yml: quality-gate job (ratchet + step summary + artifact) and check:provider-consistency in lint job
- check-provider-consistency.ts: every REGISTRY id must be a canonical provider (found krutrim half-registered → allowlisted as known pre-existing, blocks any NEW orphan)
- TDD: 9 tests (5 ratchet + 4 provider-consistency)

* feat(quality): Fase 2 anti-hallucination gates — fetch-targets, openapi-routes, deps allowlist

- check-fetch-targets: every dashboard fetch(/api/...) resolves to a real route.ts; found 7 pre-existing dashboard->route mismatches frozen as KNOWN_MISSING for triage
- check-openapi-routes: every openapi.yaml path resolves to a real route; found 1 stale spec entry (agent-bridge agents/{id}/state) frozen as KNOWN_STALE_SPEC
- check-deps: anti-slopsquatting allowlist (105 deps); new deps need explicit human-reviewed entry
- all wired into CI lint/docs jobs; TDD +12 tests (21 total across 5 gates)

* docs(quality): add quality-gates report + implementation plan to repo root

* feat(quality): Fase 3a — file-size ratchet (freeze 91 files >800 LOC, cap 800 for new)

- check-file-size.mjs: frozen files can only shrink; new files must be <= cap (kills the next 12k-line god-component)
- file-size-baseline.json: 91 files frozen at current LOC (largest 12883)
- wired into CI lint job; TDD 5 tests; --update ratchets the baseline down on shrink

* feat(quality): Fase 3b — duplication ratchet (jscpd@4, baseline 5.72%)

- check-duplication.mjs: runs jscpd@4 (pinned; v5 is an incompatible Rust rewrite) over src+open-sse, fails if duplication % rises vs frozen baseline (5.72%, measured: 1358 clones / 22967 dup lines). Targets the executor copy-paste (48/50 override execute() wholesale)
- wired into the parallel quality-gate CI job (off the lint critical path); TDD 4 tests; --update ratchets down
- snapshot now complete: coverage ~82.6%, eslint 3482 (98.5% no-explicit-any), duplication 5.72%, 91 files >800 LOC

* feat(quality): Fase 4a — anti test-masking gate

- check-test-masking.mjs: for each MODIFIED test file in a PR, flags net assert removal + new assert.ok(true) tautologies (base...HEAD diff). Directly enforces CLAUDE.md 'never weaken asserts to go green'
- wired into pr-test-policy CI job (reuses base fetch); no-op outside PR; TDD 5 tests

* feat(quality): Fase 4b — coverage ratchet (conservative floors, CI consumes merged coverage)

- quality-baseline.json: coverage.{statements,lines,functions,branches} floors (80/80/82/73, real ~82.58/82.58/84.23/75.22 with margin; tighten via --update after a green main run)
- check-quality-ratchet.mjs: --allow-missing (local quality:gate skips coverage.* without a coverage run; CI runs strict)
- ci.yml quality-gate job: needs test-coverage + downloads merged coverage-report so the ratchet enforces 'coverage cannot drop'
- TDD +1 test (6 total)

* feat(quality): Fase 6 — 8 new gates (Rule #11/#12, migrations, known-symbols, route-guard, complexity, docs-symbols, db-rules)

Deterministic gates, each freezing pre-existing violations in a documented allowlist (ratchet) so they pass now and block only NEW regressions:
- check-error-helper (Rule #12): 7 executors/handlers forwarding raw err.message frozen
- check-public-creds (Rule #11): 5 literal client_ids (Claude/Codex/Qwen/Kimi/Copilot) frozen
- check-migration-numbering: gaps 026/055 + dup 041 frozen (prevents the git-rm-deleted-migration incident)
- check-known-symbols: 93 executors conformance + 15 combo strategies + 18 translator pairs
- check-route-guard-membership (#15/#17): all 25 spawn-capable routes verified local-only (0 gaps)
- check-complexity: cyclomatic>15 / fn-length>80 ratchet (baseline 1739)
- check-docs-symbols: 30 stale doc /api refs frozen (docs hallucination)
- check-db-rules (#2/#5): 25 unexported db modules + 15 raw-SQL routes frozen
Wired into CI (lint / docs-sync-strict / quality-gate jobs). 115 TDD tests, all green. ESLint ratchet held at 3482.

* docs(quality): Phase 7 plan (security/dead-code/mutation/community tooling) — GATED to 2026-06-16

Stored, not active. 7 suggested gates + all discussed OSS/Community tools (SonarQube Community + osv-scanner + CodeQL + knip + sonarjs + type-coverage + lockfile-lint + Stryker + size-limit + axe-core + semcheck + agent-lsp + Qlty). Activation gate: do not start before 2026-06-16 (use Phases 0-6 in production for 1 week, validate in practice, then evolve).
2026-06-09 17:09:54 -03:00

238 lines
10 KiB
JavaScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
#!/usr/bin/env node
// scripts/check/check-docs-symbols.mjs
// Gate anti-alucinação (docs → código): toda referência a uma rota `/api/...` dentro de
// docs/**/*.md deve resolver para um `route.ts` real em src/app/api/. Pega endpoint
// INVENTADO/obsoleto que a IA escreve em docs/PRs descrevendo uma rota que não existe —
// o padrão recorrente das PRs de docs (ex.: oyi77) que fabricam endpoints/APIs.
//
// Complementa os outros gates anti-alucinação:
// - check-fetch-targets.mjs : fetch("/api/...") na UI → route.ts (código → código)
// - check-openapi-routes.mjs : path da openapi.yaml → route.ts (spec → código)
// - este gate : /api/... na prosa/markdown → route.ts (docs → código)
//
// LOW-NOISE por design: escopo APENAS a paths de rota `/api/...` (sinal mais alto).
// Tudo que é ruído conhecido (superfície proxy OpenAI-compat, refs a arquivos-fonte,
// APIs upstream de terceiros, placeholders) vai para IGNORE com justificativa, NÃO para
// a allowlist. A allowlist congela só drift REAL pré-existente de docs.
import fs from "node:fs";
import path from "node:path";
import { pathToFileURL } from "node:url";
const ROOT = process.cwd();
const DOCS = path.join(ROOT, "docs");
const API = path.join(ROOT, "src/app/api");
// Padrões que NÃO são rotas internas do OmniRoute (ruído estrutural, não drift).
// Adicione aqui (com justificativa) em vez da allowlist quando uma categoria gera
// falsos positivos — a allowlist é só para endpoints stale REAIS.
const IGNORE = [
/^\/api\/v1\//, // superfície OpenAI-compat (proxy), não rota interna
/^\/api\/v1beta\//, // superfície Gemini-compat (proxy)
/^\/api\/v0\//, // APIs upstream de terceiros citadas em docs de pesquisa
/^\/api\/v2\//, // idem (deployments etc.)
/^\/api\/(organizations|map-image|graphql|gql)\b/, // APIs de provedores externos documentadas
/your-/i, // placeholder de exemplo
/example/i, // placeholder de exemplo
/\.{3}/, // placeholder "..."
/\{\}/, // placeholder de param vazio
/_(POST|GET|PUT|DELETE|PATCH)$/, // refs estilo trace de rede (gql_POST)
];
// Refs a ARQUIVOS-FONTE, não a URLs (ex.: src/app/api/.../route.ts citado em prosa).
// O gate só valida URLs de rota, não caminhos de arquivo.
function isFileRef(p) {
return /\.(ts|tsx|js|mjs|jsx)$/.test(p) || /\/route$/.test(p);
}
// Refs a `/api/...` que NÃO resolvem para rota real, congeladas para triagem
// (catraca: bloqueia QUALQUER nova ref inventada em docs). Estas são achados REAIS de
// drift/alucinação em docs pré-existentes — cada uma precisa de: criar a rota, corrigir
// o path na doc, ou remover a menção. NÃO adicione novas aqui sem justificativa — esse
// é o ponto do gate. Issues de tracking devem ser abertas para cada cluster.
export const KNOWN_STALE_DOC_REFS = new Set([
// docs/reference/API_REFERENCE.md — tabela de endpoints com várias rotas obsoletas:
"/api/acp/agents/[id]", // só existe /api/acp/agents (sem [id])
"/api/acp/agents/refresh", // sem rota /refresh
"/api/admin/circuit-breaker", // admin só tem /concurrency
"/api/admin/circuit-breaker/reset", // idem
"/api/admin/rate-limits", // idem
"/api/cache/clear", // cache usa DELETE em /api/cache, não /clear
"/api/cache/reasoning/clear", // /api/cache/reasoning existe; /clear não
"/api/guardrails", // sem dir de API guardrails (feature server-side, sem rota REST)
"/api/guardrails/[id]/disable",
"/api/guardrails/[id]/enable",
"/api/guardrails/logs",
"/api/guardrails/test",
"/api/plugins/[id]/disable", // rota real usa [name] + activate/deactivate
"/api/plugins/[id]/enable", // idem
"/api/shadow", // sem dir de API shadow (shadow routing não tem rota REST)
"/api/shadow/[id]",
"/api/shadow/[id]/results",
"/api/shadow/metrics",
"/api/skills/[id]/disable", // skills tem [id] e /executions (base), não estas sub-ações
"/api/skills/[id]/enable",
"/api/skills/[id]/execute",
"/api/skills/[id]/executions",
"/api/system-info", // sem rota /system-info
// docs/research/DISCOVERY_TOOL_DESIGN.md — design doc de feature NÃO implementada:
"/api/discovery/results",
"/api/discovery/results/:id",
"/api/discovery/scan",
"/api/discovery/verify/:id",
// docs/frameworks/AGENTBRIDGE.md — state POR-AGENTE; rota real é o /state GLOBAL
// (mesmo drift congelado em check-openapi-routes.mjs::KNOWN_STALE_SPEC):
"/api/tools/agent-bridge/agents/{id}/state",
// docs/reference/ENVIRONMENT.md — endpoint UPSTREAM do provedor Blackbox Web,
// citado na descrição de env var (não é rota do OmniRoute):
"/api/chat",
// docs/ops/TUNNELS_GUIDE.md — a doc afirma EXPLICITAMENTE que este endpoint NÃO
// existe ("There is no central /api/settings/tunnels endpoint"); menção pedagógica:
"/api/settings/tunnels",
]);
function walk(dir, filter, acc = []) {
if (!fs.existsSync(dir)) return acc;
for (const e of fs.readdirSync(dir, { withFileTypes: true })) {
const p = path.join(dir, e.name);
if (e.isDirectory()) walk(p, filter, acc);
else if (filter(e.name)) acc.push(p);
}
return acc;
}
export function collectRouteFiles() {
return new Set(
walk(API, (n) => /^route\.tsx?$/.test(n)).map((p) =>
path.relative(ROOT, p).replace(/\\/g, "/")
)
);
}
/** Normaliza um segmento dinâmico ({param} / [param] / [...param] / :param) para wildcard. */
function normSeg(seg) {
if (/^\[\[?\.{3}.+\]\]?$/.test(seg)) return ""; // catch-all [...x] / [[...x]]
if (/^\{[^}]+\}$/.test(seg) || /^\[[^\]]+\]$/.test(seg) || /^:[^/]+$/.test(seg)) return " ";
return seg;
}
// /api/providers/{id}/models → src/app/api/providers/[id]/models/route.ts
// Casa por contagem de segmentos OU por prefixo (uma doc pode citar só o prefixo de
// uma rota mais profunda, ex.: /api/auth descrevendo a família /api/auth/login). Qualquer
// segmento dinâmico ([..]/{..}/:..) casa com um segmento dinâmico real.
export function resolveApiDocPathToRoute(apiPath, routeFiles) {
const segs = apiPath
.replace(/^\//, "")
.replace(/[?#].*$/, "")
.split("/")
.map(normSeg);
for (const rf of routeFiles) {
const rsegs = rf
.replace(/^src\/app\//, "")
.replace(/\/route\.tsx?$/, "")
.split("/");
const rnorm = rsegs.map((rs) => {
if (/^\[\[?\.{3}.+\]\]?$/.test(rs)) return ""; // catch-all
if (/^\[[^\]]+\]$/.test(rs)) return " "; // [param]
return rs;
});
const catchAll = rnorm.includes("");
const effLen = catchAll ? rnorm.indexOf("") : rnorm.length;
if (!catchAll && segs.length > rnorm.length) continue; // doc mais profunda que a rota
if (catchAll && segs.length < effLen) continue;
const cmpLen = Math.min(segs.length, effLen || rnorm.length);
let match = true;
for (let i = 0; i < cmpLen; i++) {
const rs = rnorm[i];
if (rs === "") break; // catch-all absorve o resto
if (!(rs === segs[i] || rs === " " || segs[i] === " ")) {
match = false;
break;
}
}
if (match) return true;
}
return false;
}
/** Limpa o path capturado: remove pontuação/ênfase de prosa, fecha brackets pendentes. */
function cleanCapturedPath(raw) {
let p = raw.replace(/[.,:;_)>]+$/, "");
const ob = (p.match(/\[/g) || []).length;
const cb = (p.match(/\]/g) || []).length;
const oc = (p.match(/\{/g) || []).length;
const cc = (p.match(/\}/g) || []).length;
if (ob !== cb || oc !== cc) {
// segmento final truncado pelo regex (bracket aberto sem fechar na prosa) → descarta
p = p.replace(/\/[^/]*[[{][^/]*$/, "");
}
return p.replace(/\/$/, ""); // remove barra final (forma de prefixo)
}
// /api/... só conta como URL quando NÃO é a cauda de um caminho de arquivo-fonte
// (src/lib/api/, @/app/api/, app/api/). O grupo 2 é o path.
const API_PATH_RE = /(^|[^A-Za-z0-9_/])(\/api\/[A-Za-z0-9_\-/{}\[\].:]+)/g;
/** Extrai os paths /api/... distintos de um arquivo markdown (forma URL, não arquivo). */
export function extractDocApiPaths(src) {
const out = new Set();
let m;
API_PATH_RE.lastIndex = 0;
while ((m = API_PATH_RE.exec(src))) {
const p = cleanCapturedPath(m[2]);
if (p && p !== "/api") out.add(p);
}
return [...out];
}
/**
* Núcleo puro/testável.
* @param {{file: string, paths: string[]}[]} docPathsByFile
* @param {Set<string>} routeFiles conjunto de "src/app/api/.../route.ts"
* @param {Set<string>} allowlist paths stale congelados
* @returns {string[]} misses no formato "file → /api/path"
*/
export function findStaleDocApiRefs(docPathsByFile, routeFiles, allowlist) {
const misses = [];
for (const { file, paths } of docPathsByFile) {
for (const p of paths) {
if (IGNORE.some((rx) => rx.test(p))) continue;
if (isFileRef(p)) continue;
if (allowlist.has(p)) continue;
if (!resolveApiDocPathToRoute(p, routeFiles)) {
misses.push(`${file}${p}`);
}
}
}
return misses;
}
function main() {
const routeFiles = collectRouteFiles();
// docs/i18n/** são espelhos auto-gerados das docs canônicas — validar só o canônico
// evita 40× de ruído duplicado (e os mirrors herdam qualquer fix do canônico).
const docFiles = walk(DOCS, (n) => /\.md$/.test(n)).filter(
(f) => !path.relative(ROOT, f).replace(/\\/g, "/").startsWith("docs/i18n/")
);
const docPathsByFile = docFiles.map((f) => ({
file: path.relative(ROOT, f).replace(/\\/g, "/"),
paths: extractDocApiPaths(fs.readFileSync(f, "utf8")),
}));
const misses = findStaleDocApiRefs(docPathsByFile, routeFiles, KNOWN_STALE_DOC_REFS);
if (misses.length) {
console.error(
`[check-docs-symbols] ${misses.length} ref(s) /api em docs sem rota real:\n` +
misses.map((m) => " ✗ " + m).join("\n") +
`\n → crie o route.ts, corrija o path na doc, ou (se for upstream/placeholder)` +
` adicione um padrão a IGNORE com justificativa. NÃO adicione à allowlist sem` +
` confirmar que é drift pré-existente real.`
);
process.exit(1);
}
console.log(
`[check-docs-symbols] OK — ${docFiles.length} docs canônicas, ` +
`${routeFiles.size} rotas conhecidas, ${KNOWN_STALE_DOC_REFS.size} stale congeladas`
);
}
if (import.meta.url === pathToFileURL(process.argv[1] || "").href) main();