mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-06 23:32:12 +03:00
* chore(release): open v3.8.19 development cycle * chore(release): sync electron lockfile to 3.8.19 * feat(quality): quality-gate ratchet + anti-hallucination/rule-enforcement guardrails (Phases 0-6) (#3471) * feat(quality): generic ratchet comparator (multi-metric, regression-only) * chore(ci): Fase 0 quality-gate fixes — reconcile coverage gate (40->60), tier npm audit, wire orphaned contract gates, re-enable cheap husky pre-commit * feat(quality): ratchet engine (collector + frozen baseline + CI job) and provider-consistency gate - collect-metrics.mjs: emits quality-metrics.json (ESLint warnings + coverage when present) - quality-baseline.json: frozen baseline (eslintWarnings=3482, regression-only) - ci.yml: quality-gate job (ratchet + step summary + artifact) and check:provider-consistency in lint job - check-provider-consistency.ts: every REGISTRY id must be a canonical provider (found krutrim half-registered → allowlisted as known pre-existing, blocks any NEW orphan) - TDD: 9 tests (5 ratchet + 4 provider-consistency) * feat(quality): Fase 2 anti-hallucination gates — fetch-targets, openapi-routes, deps allowlist - check-fetch-targets: every dashboard fetch(/api/...) resolves to a real route.ts; found 7 pre-existing dashboard->route mismatches frozen as KNOWN_MISSING for triage - check-openapi-routes: every openapi.yaml path resolves to a real route; found 1 stale spec entry (agent-bridge agents/{id}/state) frozen as KNOWN_STALE_SPEC - check-deps: anti-slopsquatting allowlist (105 deps); new deps need explicit human-reviewed entry - all wired into CI lint/docs jobs; TDD +12 tests (21 total across 5 gates) * docs(quality): add quality-gates report + implementation plan to repo root * feat(quality): Fase 3a — file-size ratchet (freeze 91 files >800 LOC, cap 800 for new) - check-file-size.mjs: frozen files can only shrink; new files must be <= cap (kills the next 12k-line god-component) - file-size-baseline.json: 91 files frozen at current LOC (largest 12883) - wired into CI lint job; TDD 5 tests; --update ratchets the baseline down on shrink * feat(quality): Fase 3b — duplication ratchet (jscpd@4, baseline 5.72%) - check-duplication.mjs: runs jscpd@4 (pinned; v5 is an incompatible Rust rewrite) over src+open-sse, fails if duplication % rises vs frozen baseline (5.72%, measured: 1358 clones / 22967 dup lines). Targets the executor copy-paste (48/50 override execute() wholesale) - wired into the parallel quality-gate CI job (off the lint critical path); TDD 4 tests; --update ratchets down - snapshot now complete: coverage ~82.6%, eslint 3482 (98.5% no-explicit-any), duplication 5.72%, 91 files >800 LOC * feat(quality): Fase 4a — anti test-masking gate - check-test-masking.mjs: for each MODIFIED test file in a PR, flags net assert removal + new assert.ok(true) tautologies (base...HEAD diff). Directly enforces CLAUDE.md 'never weaken asserts to go green' - wired into pr-test-policy CI job (reuses base fetch); no-op outside PR; TDD 5 tests * feat(quality): Fase 4b — coverage ratchet (conservative floors, CI consumes merged coverage) - quality-baseline.json: coverage.{statements,lines,functions,branches} floors (80/80/82/73, real ~82.58/82.58/84.23/75.22 with margin; tighten via --update after a green main run) - check-quality-ratchet.mjs: --allow-missing (local quality:gate skips coverage.* without a coverage run; CI runs strict) - ci.yml quality-gate job: needs test-coverage + downloads merged coverage-report so the ratchet enforces 'coverage cannot drop' - TDD +1 test (6 total) * feat(quality): Fase 6 — 8 new gates (Rule #11/#12, migrations, known-symbols, route-guard, complexity, docs-symbols, db-rules) Deterministic gates, each freezing pre-existing violations in a documented allowlist (ratchet) so they pass now and block only NEW regressions: - check-error-helper (Rule #12): 7 executors/handlers forwarding raw err.message frozen - check-public-creds (Rule #11): 5 literal client_ids (Claude/Codex/Qwen/Kimi/Copilot) frozen - check-migration-numbering: gaps 026/055 + dup 041 frozen (prevents the git-rm-deleted-migration incident) - check-known-symbols: 93 executors conformance + 15 combo strategies + 18 translator pairs - check-route-guard-membership (#15/#17): all 25 spawn-capable routes verified local-only (0 gaps) - check-complexity: cyclomatic>15 / fn-length>80 ratchet (baseline 1739) - check-docs-symbols: 30 stale doc /api refs frozen (docs hallucination) - check-db-rules (#2/#5): 25 unexported db modules + 15 raw-SQL routes frozen Wired into CI (lint / docs-sync-strict / quality-gate jobs). 115 TDD tests, all green. ESLint ratchet held at 3482. * docs(quality): Phase 7 plan (security/dead-code/mutation/community tooling) — GATED to 2026-06-16 Stored, not active. 7 suggested gates + all discussed OSS/Community tools (SonarQube Community + osv-scanner + CodeQL + knip + sonarjs + type-coverage + lockfile-lint + Stryker + size-limit + axe-core + semcheck + agent-lsp + Qlty). Activation gate: do not start before 2026-06-16 (use Phases 0-6 in production for 1 week, validate in practice, then evolve). * docs(quality): Phase 6A critical-audit plan + Phase 7 additions — gated to 2026-06-16 (#3530) PLANO-QUALITY-GATES-FASE6A.md (12-task audit of Phases 0-6: orphan tests, stale-allowlist enforcement, scope gaps) + Phase 7 additions (gitleaks, actionlint+zizmor, license compliance). Both stored, activation gated to 2026-06-16. Tasks 6A.1/6A.2 were fast-tracked separately (#3536). * feat(quality): 6A.1+6A.2 — test-discovery gate, 135 orphan tests re-wired, 2 production bug fixes, vitest in CI (#3536) check-test-discovery gate (TDD; 195 orphans found, 135 re-wired into the node runner, 60 frozen+annotated). Triage fixed 2 real production bugs: missing BYPASS_PREFIX_NOT_ALLOWED zod refine (spawn-capable prefixes accepted into the bypass list, Hard Rules #15/#17) and resetDbInstance not firing stateReset resetters (stale schema memo → 503 instead of 403; also hit backup-restore). New test-vitest CI job: test:vitest blocking (146/146), test:vitest:ui informational (14 pre-existing fails, triage 2026-06-16). * chore: ignore generated yt-downloader artifact files Add dated yt-downloader output files to .gitignore to prevent local automation artifacts from being accidentally committed. * chore(quality): green-light the quality-gate — conscious file-size + eslintWarnings re-baselines (#3538) file-size: 9 files frozen at current sizes (v3.8.18-era growth + core.ts +7 from #3536 fix). eslintWarnings 3482→3501: the published v3.8.18 tag already measures 3501 (delta predates the quality-gate job); v3.8.19 cycle is neutral. Reduction + --require-tighten = Phase 6A (2026-06-16). * fix(check): exclude internal planning docs (docs/superpowers/) from the docs-symbols gate docs/superpowers/plans/*.md are historical implementation-plan snapshots that may cite planned/abandoned routes — not claims about the current code. Three such refs entered during the v3.8.18 cycle, before this gate was on the pipeline, and would have blocked the v3.8.19 release merge. * chore(release): v3.8.19 — 2026-06-09 CHANGELOG section for the quality-infrastructure release (7 commits, 1:1 coverage), [3.8.18] label corrected to its real release date, local prompt artifacts ignored. * test: hermetic auth context for 2 re-wired suites + real headroom on the breaker reset-timeout flake CI shards exposed what the dev DATA_DIR was masking locally: detect.test.ts and managementCliToken.test.ts asserted 401/403/reject outcomes that only exist when login protection is configured — on a fresh CI DB isAuthRequired() is false and the policy anonymous-allows. Both now create an isolated DATA_DIR with requireLogin+password (the established pattern). observability-fase04: the breaker reset-timeout test ran with a 5ms margin (resetTimeout 10 / sleep 15) — lazy HALF_OPEN refresh under shard contention flipped the first OPEN assert. Now 250/300ms. * test: align bypass-prefix schema test to the restored layer-1 contract + real waitFor headroom appearance-widget-settings-schema asserted that /api/cli-tools/runtime/ was ACCEPTED into the bypass list — written against the buggy schema (missing BYPASS_PREFIX_NOT_ALLOWED refine, restored in #3536) and consecrating the bug the AC-8 orphan test guards against. Split into accept-safe + reject-spawn-capable cases. chatcore waitFor ceiling 1500→10000ms (green runs return immediately; observed 1580ms expiry on 2-core CI runners). * test(chatcore): fix structurally-broken pending-detail predicate (flatten before find) pendingRequests.details[connectionId] is Record<modelKey, PendingRequestDetail[]> — the upstream-timeout test's waitFor tested each ARRAY's .providerRequest (always undefined), so it could never resolve and expired (failed on 3 CI jobs; reproduced deterministically isolated, including at the published v3.8.18 tag). Flatten to the actual details + declare the call_log_pipeline_enabled dependency explicitly + waitFor ceiling with real CI headroom. * chore(quality): re-baseline coverage floors to the honest post-re-wire denominator + changelog coverage for the stabilization commits The 135 re-wired tests import modules that were never loaded before, so the c8 denominator grew: the old ~82.5% was inflated by never-imported modules being invisible. CI merged coverage now measures 78.4/78.4/83.84/75.73 — floors set ~2pt below (76.5/76.5; functions/branches floors already hold). Tightening via --require-tighten is Phase 6A work (2026-06-16).
242 lines
10 KiB
JavaScript
242 lines
10 KiB
JavaScript
#!/usr/bin/env node
|
||
// scripts/check/check-docs-symbols.mjs
|
||
// Gate anti-alucinação (docs → código): toda referência a uma rota `/api/...` dentro de
|
||
// docs/**/*.md deve resolver para um `route.ts` real em src/app/api/. Pega endpoint
|
||
// INVENTADO/obsoleto que a IA escreve em docs/PRs descrevendo uma rota que não existe —
|
||
// o padrão recorrente das PRs de docs (ex.: oyi77) que fabricam endpoints/APIs.
|
||
//
|
||
// Complementa os outros gates anti-alucinação:
|
||
// - check-fetch-targets.mjs : fetch("/api/...") na UI → route.ts (código → código)
|
||
// - check-openapi-routes.mjs : path da openapi.yaml → route.ts (spec → código)
|
||
// - este gate : /api/... na prosa/markdown → route.ts (docs → código)
|
||
//
|
||
// LOW-NOISE por design: escopo APENAS a paths de rota `/api/...` (sinal mais alto).
|
||
// Tudo que é ruído conhecido (superfície proxy OpenAI-compat, refs a arquivos-fonte,
|
||
// APIs upstream de terceiros, placeholders) vai para IGNORE com justificativa, NÃO para
|
||
// a allowlist. A allowlist congela só drift REAL pré-existente de docs.
|
||
import fs from "node:fs";
|
||
import path from "node:path";
|
||
import { pathToFileURL } from "node:url";
|
||
|
||
const ROOT = process.cwd();
|
||
const DOCS = path.join(ROOT, "docs");
|
||
const API = path.join(ROOT, "src/app/api");
|
||
|
||
// Padrões que NÃO são rotas internas do OmniRoute (ruído estrutural, não drift).
|
||
// Adicione aqui (com justificativa) em vez da allowlist quando uma categoria gera
|
||
// falsos positivos — a allowlist é só para endpoints stale REAIS.
|
||
const IGNORE = [
|
||
/^\/api\/v1\//, // superfície OpenAI-compat (proxy), não rota interna
|
||
/^\/api\/v1beta\//, // superfície Gemini-compat (proxy)
|
||
/^\/api\/v0\//, // APIs upstream de terceiros citadas em docs de pesquisa
|
||
/^\/api\/v2\//, // idem (deployments etc.)
|
||
/^\/api\/(organizations|map-image|graphql|gql)\b/, // APIs de provedores externos documentadas
|
||
/your-/i, // placeholder de exemplo
|
||
/example/i, // placeholder de exemplo
|
||
/\.{3}/, // placeholder "..."
|
||
/\{\}/, // placeholder de param vazio
|
||
/_(POST|GET|PUT|DELETE|PATCH)$/, // refs estilo trace de rede (gql_POST)
|
||
];
|
||
|
||
// Refs a ARQUIVOS-FONTE, não a URLs (ex.: src/app/api/.../route.ts citado em prosa).
|
||
// O gate só valida URLs de rota, não caminhos de arquivo.
|
||
function isFileRef(p) {
|
||
return /\.(ts|tsx|js|mjs|jsx)$/.test(p) || /\/route$/.test(p);
|
||
}
|
||
|
||
// Refs a `/api/...` que NÃO resolvem para rota real, congeladas para triagem
|
||
// (catraca: bloqueia QUALQUER nova ref inventada em docs). Estas são achados REAIS de
|
||
// drift/alucinação em docs pré-existentes — cada uma precisa de: criar a rota, corrigir
|
||
// o path na doc, ou remover a menção. NÃO adicione novas aqui sem justificativa — esse
|
||
// é o ponto do gate. Issues de tracking devem ser abertas para cada cluster.
|
||
export const KNOWN_STALE_DOC_REFS = new Set([
|
||
// docs/reference/API_REFERENCE.md — tabela de endpoints com várias rotas obsoletas:
|
||
"/api/acp/agents/[id]", // só existe /api/acp/agents (sem [id])
|
||
"/api/acp/agents/refresh", // sem rota /refresh
|
||
"/api/admin/circuit-breaker", // admin só tem /concurrency
|
||
"/api/admin/circuit-breaker/reset", // idem
|
||
"/api/admin/rate-limits", // idem
|
||
"/api/cache/clear", // cache usa DELETE em /api/cache, não /clear
|
||
"/api/cache/reasoning/clear", // /api/cache/reasoning existe; /clear não
|
||
"/api/guardrails", // sem dir de API guardrails (feature server-side, sem rota REST)
|
||
"/api/guardrails/[id]/disable",
|
||
"/api/guardrails/[id]/enable",
|
||
"/api/guardrails/logs",
|
||
"/api/guardrails/test",
|
||
"/api/plugins/[id]/disable", // rota real usa [name] + activate/deactivate
|
||
"/api/plugins/[id]/enable", // idem
|
||
"/api/shadow", // sem dir de API shadow (shadow routing não tem rota REST)
|
||
"/api/shadow/[id]",
|
||
"/api/shadow/[id]/results",
|
||
"/api/shadow/metrics",
|
||
"/api/skills/[id]/disable", // skills tem [id] e /executions (base), não estas sub-ações
|
||
"/api/skills/[id]/enable",
|
||
"/api/skills/[id]/execute",
|
||
"/api/skills/[id]/executions",
|
||
"/api/system-info", // sem rota /system-info
|
||
// docs/research/DISCOVERY_TOOL_DESIGN.md — design doc de feature NÃO implementada:
|
||
"/api/discovery/results",
|
||
"/api/discovery/results/:id",
|
||
"/api/discovery/scan",
|
||
"/api/discovery/verify/:id",
|
||
// docs/frameworks/AGENTBRIDGE.md — state POR-AGENTE; rota real é o /state GLOBAL
|
||
// (mesmo drift congelado em check-openapi-routes.mjs::KNOWN_STALE_SPEC):
|
||
"/api/tools/agent-bridge/agents/{id}/state",
|
||
// docs/reference/ENVIRONMENT.md — endpoint UPSTREAM do provedor Blackbox Web,
|
||
// citado na descrição de env var (não é rota do OmniRoute):
|
||
"/api/chat",
|
||
// docs/ops/TUNNELS_GUIDE.md — a doc afirma EXPLICITAMENTE que este endpoint NÃO
|
||
// existe ("There is no central /api/settings/tunnels endpoint"); menção pedagógica:
|
||
"/api/settings/tunnels",
|
||
]);
|
||
|
||
function walk(dir, filter, acc = []) {
|
||
if (!fs.existsSync(dir)) return acc;
|
||
for (const e of fs.readdirSync(dir, { withFileTypes: true })) {
|
||
const p = path.join(dir, e.name);
|
||
if (e.isDirectory()) walk(p, filter, acc);
|
||
else if (filter(e.name)) acc.push(p);
|
||
}
|
||
return acc;
|
||
}
|
||
|
||
export function collectRouteFiles() {
|
||
return new Set(
|
||
walk(API, (n) => /^route\.tsx?$/.test(n)).map((p) =>
|
||
path.relative(ROOT, p).replace(/\\/g, "/")
|
||
)
|
||
);
|
||
}
|
||
|
||
/** Normaliza um segmento dinâmico ({param} / [param] / [...param] / :param) para wildcard. */
|
||
function normSeg(seg) {
|
||
if (/^\[\[?\.{3}.+\]\]?$/.test(seg)) return ""; // catch-all [...x] / [[...x]]
|
||
if (/^\{[^}]+\}$/.test(seg) || /^\[[^\]]+\]$/.test(seg) || /^:[^/]+$/.test(seg)) return " ";
|
||
return seg;
|
||
}
|
||
|
||
// /api/providers/{id}/models → src/app/api/providers/[id]/models/route.ts
|
||
// Casa por contagem de segmentos OU por prefixo (uma doc pode citar só o prefixo de
|
||
// uma rota mais profunda, ex.: /api/auth descrevendo a família /api/auth/login). Qualquer
|
||
// segmento dinâmico ([..]/{..}/:..) casa com um segmento dinâmico real.
|
||
export function resolveApiDocPathToRoute(apiPath, routeFiles) {
|
||
const segs = apiPath
|
||
.replace(/^\//, "")
|
||
.replace(/[?#].*$/, "")
|
||
.split("/")
|
||
.map(normSeg);
|
||
for (const rf of routeFiles) {
|
||
const rsegs = rf
|
||
.replace(/^src\/app\//, "")
|
||
.replace(/\/route\.tsx?$/, "")
|
||
.split("/");
|
||
const rnorm = rsegs.map((rs) => {
|
||
if (/^\[\[?\.{3}.+\]\]?$/.test(rs)) return ""; // catch-all
|
||
if (/^\[[^\]]+\]$/.test(rs)) return " "; // [param]
|
||
return rs;
|
||
});
|
||
const catchAll = rnorm.includes("");
|
||
const effLen = catchAll ? rnorm.indexOf("") : rnorm.length;
|
||
if (!catchAll && segs.length > rnorm.length) continue; // doc mais profunda que a rota
|
||
if (catchAll && segs.length < effLen) continue;
|
||
const cmpLen = Math.min(segs.length, effLen || rnorm.length);
|
||
let match = true;
|
||
for (let i = 0; i < cmpLen; i++) {
|
||
const rs = rnorm[i];
|
||
if (rs === "") break; // catch-all absorve o resto
|
||
if (!(rs === segs[i] || rs === " " || segs[i] === " ")) {
|
||
match = false;
|
||
break;
|
||
}
|
||
}
|
||
if (match) return true;
|
||
}
|
||
return false;
|
||
}
|
||
|
||
/** Limpa o path capturado: remove pontuação/ênfase de prosa, fecha brackets pendentes. */
|
||
function cleanCapturedPath(raw) {
|
||
let p = raw.replace(/[.,:;_)>]+$/, "");
|
||
const ob = (p.match(/\[/g) || []).length;
|
||
const cb = (p.match(/\]/g) || []).length;
|
||
const oc = (p.match(/\{/g) || []).length;
|
||
const cc = (p.match(/\}/g) || []).length;
|
||
if (ob !== cb || oc !== cc) {
|
||
// segmento final truncado pelo regex (bracket aberto sem fechar na prosa) → descarta
|
||
p = p.replace(/\/[^/]*[[{][^/]*$/, "");
|
||
}
|
||
return p.replace(/\/$/, ""); // remove barra final (forma de prefixo)
|
||
}
|
||
|
||
// /api/... só conta como URL quando NÃO é a cauda de um caminho de arquivo-fonte
|
||
// (src/lib/api/, @/app/api/, app/api/). O grupo 2 é o path.
|
||
const API_PATH_RE = /(^|[^A-Za-z0-9_/])(\/api\/[A-Za-z0-9_\-/{}\[\].:]+)/g;
|
||
|
||
/** Extrai os paths /api/... distintos de um arquivo markdown (forma URL, não arquivo). */
|
||
export function extractDocApiPaths(src) {
|
||
const out = new Set();
|
||
let m;
|
||
API_PATH_RE.lastIndex = 0;
|
||
while ((m = API_PATH_RE.exec(src))) {
|
||
const p = cleanCapturedPath(m[2]);
|
||
if (p && p !== "/api") out.add(p);
|
||
}
|
||
return [...out];
|
||
}
|
||
|
||
/**
|
||
* Núcleo puro/testável.
|
||
* @param {{file: string, paths: string[]}[]} docPathsByFile
|
||
* @param {Set<string>} routeFiles conjunto de "src/app/api/.../route.ts"
|
||
* @param {Set<string>} allowlist paths stale congelados
|
||
* @returns {string[]} misses no formato "file → /api/path"
|
||
*/
|
||
export function findStaleDocApiRefs(docPathsByFile, routeFiles, allowlist) {
|
||
const misses = [];
|
||
for (const { file, paths } of docPathsByFile) {
|
||
for (const p of paths) {
|
||
if (IGNORE.some((rx) => rx.test(p))) continue;
|
||
if (isFileRef(p)) continue;
|
||
if (allowlist.has(p)) continue;
|
||
if (!resolveApiDocPathToRoute(p, routeFiles)) {
|
||
misses.push(`${file} → ${p}`);
|
||
}
|
||
}
|
||
}
|
||
return misses;
|
||
}
|
||
|
||
function main() {
|
||
const routeFiles = collectRouteFiles();
|
||
// docs/i18n/** são espelhos auto-gerados das docs canônicas — validar só o canônico
|
||
// evita 40× de ruído duplicado (e os mirrors herdam qualquer fix do canônico).
|
||
// docs/superpowers/** são planos internos de implementação (snapshots históricos
|
||
// de intenção — podem citar rotas planejadas/abandonadas), não claims sobre o
|
||
// código atual; fora do escopo do gate (drift surgiu no ciclo v3.8.18).
|
||
const docFiles = walk(DOCS, (n) => /\.md$/.test(n)).filter((f) => {
|
||
const rel = path.relative(ROOT, f).replace(/\\/g, "/");
|
||
return !rel.startsWith("docs/i18n/") && !rel.startsWith("docs/superpowers/");
|
||
});
|
||
const docPathsByFile = docFiles.map((f) => ({
|
||
file: path.relative(ROOT, f).replace(/\\/g, "/"),
|
||
paths: extractDocApiPaths(fs.readFileSync(f, "utf8")),
|
||
}));
|
||
const misses = findStaleDocApiRefs(docPathsByFile, routeFiles, KNOWN_STALE_DOC_REFS);
|
||
if (misses.length) {
|
||
console.error(
|
||
`[check-docs-symbols] ${misses.length} ref(s) /api em docs sem rota real:\n` +
|
||
misses.map((m) => " ✗ " + m).join("\n") +
|
||
`\n → crie o route.ts, corrija o path na doc, ou (se for upstream/placeholder)` +
|
||
` adicione um padrão a IGNORE com justificativa. NÃO adicione à allowlist sem` +
|
||
` confirmar que é drift pré-existente real.`
|
||
);
|
||
process.exit(1);
|
||
}
|
||
console.log(
|
||
`[check-docs-symbols] OK — ${docFiles.length} docs canônicas, ` +
|
||
`${routeFiles.size} rotas conhecidas, ${KNOWN_STALE_DOC_REFS.size} stale congeladas`
|
||
);
|
||
}
|
||
|
||
if (import.meta.url === pathToFileURL(process.argv[1] || "").href) main();
|