Files
OmniRoute/scripts/check/check-db-rules.mjs
Diego Rodrigues de Sa e Souza 68e4d0c599 Release v3.8.19 (#3526)
* chore(release): open v3.8.19 development cycle

* chore(release): sync electron lockfile to 3.8.19

* feat(quality): quality-gate ratchet + anti-hallucination/rule-enforcement guardrails (Phases 0-6) (#3471)

* feat(quality): generic ratchet comparator (multi-metric, regression-only)

* chore(ci): Fase 0 quality-gate fixes — reconcile coverage gate (40->60), tier npm audit, wire orphaned contract gates, re-enable cheap husky pre-commit

* feat(quality): ratchet engine (collector + frozen baseline + CI job) and provider-consistency gate

- collect-metrics.mjs: emits quality-metrics.json (ESLint warnings + coverage when present)
- quality-baseline.json: frozen baseline (eslintWarnings=3482, regression-only)
- ci.yml: quality-gate job (ratchet + step summary + artifact) and check:provider-consistency in lint job
- check-provider-consistency.ts: every REGISTRY id must be a canonical provider (found krutrim half-registered → allowlisted as known pre-existing, blocks any NEW orphan)
- TDD: 9 tests (5 ratchet + 4 provider-consistency)

* feat(quality): Fase 2 anti-hallucination gates — fetch-targets, openapi-routes, deps allowlist

- check-fetch-targets: every dashboard fetch(/api/...) resolves to a real route.ts; found 7 pre-existing dashboard->route mismatches frozen as KNOWN_MISSING for triage
- check-openapi-routes: every openapi.yaml path resolves to a real route; found 1 stale spec entry (agent-bridge agents/{id}/state) frozen as KNOWN_STALE_SPEC
- check-deps: anti-slopsquatting allowlist (105 deps); new deps need explicit human-reviewed entry
- all wired into CI lint/docs jobs; TDD +12 tests (21 total across 5 gates)

* docs(quality): add quality-gates report + implementation plan to repo root

* feat(quality): Fase 3a — file-size ratchet (freeze 91 files >800 LOC, cap 800 for new)

- check-file-size.mjs: frozen files can only shrink; new files must be <= cap (kills the next 12k-line god-component)
- file-size-baseline.json: 91 files frozen at current LOC (largest 12883)
- wired into CI lint job; TDD 5 tests; --update ratchets the baseline down on shrink

* feat(quality): Fase 3b — duplication ratchet (jscpd@4, baseline 5.72%)

- check-duplication.mjs: runs jscpd@4 (pinned; v5 is an incompatible Rust rewrite) over src+open-sse, fails if duplication % rises vs frozen baseline (5.72%, measured: 1358 clones / 22967 dup lines). Targets the executor copy-paste (48/50 override execute() wholesale)
- wired into the parallel quality-gate CI job (off the lint critical path); TDD 4 tests; --update ratchets down
- snapshot now complete: coverage ~82.6%, eslint 3482 (98.5% no-explicit-any), duplication 5.72%, 91 files >800 LOC

* feat(quality): Fase 4a — anti test-masking gate

- check-test-masking.mjs: for each MODIFIED test file in a PR, flags net assert removal + new assert.ok(true) tautologies (base...HEAD diff). Directly enforces CLAUDE.md 'never weaken asserts to go green'
- wired into pr-test-policy CI job (reuses base fetch); no-op outside PR; TDD 5 tests

* feat(quality): Fase 4b — coverage ratchet (conservative floors, CI consumes merged coverage)

- quality-baseline.json: coverage.{statements,lines,functions,branches} floors (80/80/82/73, real ~82.58/82.58/84.23/75.22 with margin; tighten via --update after a green main run)
- check-quality-ratchet.mjs: --allow-missing (local quality:gate skips coverage.* without a coverage run; CI runs strict)
- ci.yml quality-gate job: needs test-coverage + downloads merged coverage-report so the ratchet enforces 'coverage cannot drop'
- TDD +1 test (6 total)

* feat(quality): Fase 6 — 8 new gates (Rule #11/#12, migrations, known-symbols, route-guard, complexity, docs-symbols, db-rules)

Deterministic gates, each freezing pre-existing violations in a documented allowlist (ratchet) so they pass now and block only NEW regressions:
- check-error-helper (Rule #12): 7 executors/handlers forwarding raw err.message frozen
- check-public-creds (Rule #11): 5 literal client_ids (Claude/Codex/Qwen/Kimi/Copilot) frozen
- check-migration-numbering: gaps 026/055 + dup 041 frozen (prevents the git-rm-deleted-migration incident)
- check-known-symbols: 93 executors conformance + 15 combo strategies + 18 translator pairs
- check-route-guard-membership (#15/#17): all 25 spawn-capable routes verified local-only (0 gaps)
- check-complexity: cyclomatic>15 / fn-length>80 ratchet (baseline 1739)
- check-docs-symbols: 30 stale doc /api refs frozen (docs hallucination)
- check-db-rules (#2/#5): 25 unexported db modules + 15 raw-SQL routes frozen
Wired into CI (lint / docs-sync-strict / quality-gate jobs). 115 TDD tests, all green. ESLint ratchet held at 3482.

* docs(quality): Phase 7 plan (security/dead-code/mutation/community tooling) — GATED to 2026-06-16

Stored, not active. 7 suggested gates + all discussed OSS/Community tools (SonarQube Community + osv-scanner + CodeQL + knip + sonarjs + type-coverage + lockfile-lint + Stryker + size-limit + axe-core + semcheck + agent-lsp + Qlty). Activation gate: do not start before 2026-06-16 (use Phases 0-6 in production for 1 week, validate in practice, then evolve).

* docs(quality): Phase 6A critical-audit plan + Phase 7 additions — gated to 2026-06-16 (#3530)

PLANO-QUALITY-GATES-FASE6A.md (12-task audit of Phases 0-6: orphan tests, stale-allowlist enforcement, scope gaps) + Phase 7 additions (gitleaks, actionlint+zizmor, license compliance). Both stored, activation gated to 2026-06-16. Tasks 6A.1/6A.2 were fast-tracked separately (#3536).

* feat(quality): 6A.1+6A.2 — test-discovery gate, 135 orphan tests re-wired, 2 production bug fixes, vitest in CI (#3536)

check-test-discovery gate (TDD; 195 orphans found, 135 re-wired into the node runner, 60 frozen+annotated). Triage fixed 2 real production bugs: missing BYPASS_PREFIX_NOT_ALLOWED zod refine (spawn-capable prefixes accepted into the bypass list, Hard Rules #15/#17) and resetDbInstance not firing stateReset resetters (stale schema memo → 503 instead of 403; also hit backup-restore). New test-vitest CI job: test:vitest blocking (146/146), test:vitest:ui informational (14 pre-existing fails, triage 2026-06-16).

* chore: ignore generated yt-downloader artifact files

Add dated yt-downloader output files to .gitignore to prevent
local automation artifacts from being accidentally committed.

* chore(quality): green-light the quality-gate — conscious file-size + eslintWarnings re-baselines (#3538)

file-size: 9 files frozen at current sizes (v3.8.18-era growth + core.ts +7 from #3536 fix). eslintWarnings 3482→3501: the published v3.8.18 tag already measures 3501 (delta predates the quality-gate job); v3.8.19 cycle is neutral. Reduction + --require-tighten = Phase 6A (2026-06-16).

* fix(check): exclude internal planning docs (docs/superpowers/) from the docs-symbols gate

docs/superpowers/plans/*.md are historical implementation-plan snapshots that
may cite planned/abandoned routes — not claims about the current code. Three
such refs entered during the v3.8.18 cycle, before this gate was on the
pipeline, and would have blocked the v3.8.19 release merge.

* chore(release): v3.8.19 — 2026-06-09

CHANGELOG section for the quality-infrastructure release (7 commits, 1:1
coverage), [3.8.18] label corrected to its real release date, local prompt
artifacts ignored.

* test: hermetic auth context for 2 re-wired suites + real headroom on the breaker reset-timeout flake

CI shards exposed what the dev DATA_DIR was masking locally: detect.test.ts
and managementCliToken.test.ts asserted 401/403/reject outcomes that only
exist when login protection is configured — on a fresh CI DB isAuthRequired()
is false and the policy anonymous-allows. Both now create an isolated
DATA_DIR with requireLogin+password (the established pattern).

observability-fase04: the breaker reset-timeout test ran with a 5ms margin
(resetTimeout 10 / sleep 15) — lazy HALF_OPEN refresh under shard contention
flipped the first OPEN assert. Now 250/300ms.

* test: align bypass-prefix schema test to the restored layer-1 contract + real waitFor headroom

appearance-widget-settings-schema asserted that /api/cli-tools/runtime/ was
ACCEPTED into the bypass list — written against the buggy schema (missing
BYPASS_PREFIX_NOT_ALLOWED refine, restored in #3536) and consecrating the
bug the AC-8 orphan test guards against. Split into accept-safe +
reject-spawn-capable cases. chatcore waitFor ceiling 1500→10000ms (green
runs return immediately; observed 1580ms expiry on 2-core CI runners).

* test(chatcore): fix structurally-broken pending-detail predicate (flatten before find)

pendingRequests.details[connectionId] is Record<modelKey, PendingRequestDetail[]>
— the upstream-timeout test's waitFor tested each ARRAY's .providerRequest
(always undefined), so it could never resolve and expired (failed on 3 CI jobs;
reproduced deterministically isolated, including at the published v3.8.18 tag).
Flatten to the actual details + declare the call_log_pipeline_enabled dependency
explicitly + waitFor ceiling with real CI headroom.

* chore(quality): re-baseline coverage floors to the honest post-re-wire denominator + changelog coverage for the stabilization commits

The 135 re-wired tests import modules that were never loaded before, so the
c8 denominator grew: the old ~82.5% was inflated by never-imported modules
being invisible. CI merged coverage now measures 78.4/78.4/83.84/75.73 —
floors set ~2pt below (76.5/76.5; functions/branches floors already hold).
Tightening via --require-tighten is Phase 6A work (2026-06-16).
2026-06-09 22:57:12 -03:00

253 lines
11 KiB
JavaScript
Raw Blame History

This file contains invisible Unicode characters
This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
#!/usr/bin/env node
// scripts/check/check-db-rules.mjs
// Gate de convenções de banco (CLAUDE.md Hard Rules #2 e #5). Três verificações:
// (a) Todo módulo de domínio em src/lib/db/*.ts deve ser re-exportado por
// src/lib/localDb.ts (camada de compat). Um módulo db NOVO que não é
// re-exportado (e não está congelado) falha — força a decisão consciente
// de expor ou justificar (Hard Rule #2).
// (b) src/lib/localDb.ts é APENAS camada de re-export: nada de lógica
// (function/class/arrow de negócio). Mata o anti-padrão de "só uma
// funçãozinha aqui" que vira regra de negócio fora dos módulos db/.
// (c) Nenhum SQL cru em src/app/api/**/route.ts ou open-sse/handlers/*.ts.
// SQL deve viver em src/lib/db/ (Hard Rule #5). Ofensores pré-existentes
// são congelados; QUALQUER novo SQL cru em rota/handler falha.
import fs from "node:fs";
import path from "node:path";
import { pathToFileURL } from "node:url";
const cwd = process.cwd();
const DB_DIR = path.join(cwd, "src/lib/db");
const LOCAL_DB = path.join(cwd, "src/lib/localDb.ts");
const API_DIR = path.join(cwd, "src/app/api");
const HANDLERS_DIR = path.join(cwd, "open-sse/handlers");
// (a) Módulos db/ que NÃO são re-exportados por localDb.ts hoje. Congelados
// para a catraca ficar verde e bloquear QUALQUER módulo novo não re-exportado.
// CADA UM é dívida: ou é consumido por import direto de "@/lib/db/X" (legítimo,
// não precisa de re-export) ou deveria ser re-exportado. NÃO adicione novos aqui
// sem justificativa — esse é o ponto do gate (Hard Rule #2).
const KNOWN_UNEXPORTED = new Set([
"_rowTypes", // só tipos de linha (sem runtime API), consumido localmente pelos CRUDs F2
"cleanup", // rotina de manutenção, chamada por jobs/rotas via import direto
"cliToolState", // estado de CLI tools, import direto pelos consumidores
"comboForecast", // previsão de combo, import direto
"commandCodeAuth", // auth de command-code, import direto
"compression", // núcleo de compressão, import direto
"compressionScheduler", // scheduler, import direto
"detailedLogs", // logs detalhados, import direto
"discovery", // discovery de modelos, import direto
"domainState", // estado de domínio/circuit breaker, import direto
"encryption", // util de cripto at-rest, import direto
"healthCheck", // health check de DB, import direto
"jsonMigration", // migração JSON→SQLite (one-shot), import direto
"migrationRunner", // runner de migrations, import direto
"notion", // integração Notion, import direto
"obsidian", // integração Obsidian, import direto
"pluginMetrics", // métricas de plugin, import direto
"prompts", // prompts salvos, import direto
"providerStats", // stats de provider, import direto
"recovery", // recuperação de DB, import direto
"secrets", // secrets store, import direto
"serviceModels", // modelos de serviços embutidos, import direto
"stateReset", // reset de estado de resiliência, import direto
"stats", // agregações de stats, import direto
"tierConfig", // config de tier, import direto
]);
// (c) Ofensores de SQL cru PRÉ-EXISTENTES em rotas/handlers. Congelados para a
// catraca ficar verde e bloquear QUALQUER nova rota/handler com SQL inline.
// CADA UM é dívida da Hard Rule #5: mover para um módulo src/lib/db/. NÃO
// adicione novos aqui sem justificativa — crie/estenda um módulo db/ em vez disso.
// (Chaves = caminho relativo POSIX a partir da raiz do repo.)
const KNOWN_RAW_SQL = new Set([
"src/app/api/analytics/auto-routing/route.ts", // SELECT … FROM usage_logs
"src/app/api/cache/entries/route.ts", // semantic_cache COUNT/DELETE inline
"src/app/api/db-backups/exportAll/route.ts", // SELECT key_value/combos/connections/keys
"src/app/api/db-backups/import/route.ts", // SELECT sqlite_master + COUNTs
"src/app/api/gamification/federation/leaderboard/route.ts", // SELECT community_servers
"src/app/api/gamification/federation/score/route.ts", // SELECT community_servers
"src/app/api/logs/export/route.ts", // SELECT de proxy_logs
"src/app/api/oauth/cursor/auto-import/route.ts", // SELECT no itemTable do Cursor (DB externo)
"src/app/api/oauth/kiro/auto-import/route.ts", // SELECT no SQLite do Kiro (DB externo)
"src/app/api/provider-metrics/route.ts", // SELECT … FROM call_logs (agregação)
"src/app/api/search/stats/route.ts", // SELECT … FROM call_logs
"src/app/api/settings/export-json/route.ts", // SELECT * de usage_history/domain_*
"src/app/api/skills/[id]/route.ts", // UPDATE skills SET dinâmico
"src/app/api/usage/analytics/route.ts", // SELECT … FROM usage_history/daily_usage_summary
"src/app/api/v1/search/analytics/route.ts", // SELECT … FROM call_logs (request_type=search)
]);
// Módulos sempre excluídos da checagem (a): não são domínio re-exportável.
const DB_MODULE_EXCLUDE = new Set(["core", "localDb", "index"]);
function walk(dir, acc = []) {
if (!fs.existsSync(dir)) return acc;
for (const e of fs.readdirSync(dir, { withFileTypes: true })) {
const p = path.join(dir, e.name);
if (e.isDirectory()) walk(p, acc);
else acc.push(p);
}
return acc;
}
// Lista os módulos de domínio em src/lib/db (top-level *.ts), excluindo
// core/localDb/index, *.d.ts e qualquer subdiretório (migrations/, adapters/, __tests__/).
export function collectDbModules(dbDir = DB_DIR) {
if (!fs.existsSync(dbDir)) return [];
return fs
.readdirSync(dbDir, { withFileTypes: true })
.filter((e) => e.isFile() && /\.ts$/.test(e.name) && !/\.d\.ts$/.test(e.name))
.map((e) => e.name.replace(/\.ts$/, ""))
.filter((name) => !DB_MODULE_EXCLUDE.has(name))
.sort();
}
// Extrai os nomes de módulo re-exportados de localDb.ts a partir de
// `... from "./db/X"` (cobre export {…}, export * e export type {…}).
export function extractReexportedModules(localDbSource) {
const re = /from\s+["']\.\/db\/([A-Za-z0-9_]+)["']/g;
const out = new Set();
let m;
while ((m = re.exec(localDbSource))) out.add(m[1]);
return out;
}
// (a) Módulos db/ que não são re-exportados e não estão congelados.
export function findMissingReexports(dbModules, reexported, allowlist = KNOWN_UNEXPORTED) {
return dbModules.filter((mod) => !reexported.has(mod) && !allowlist.has(mod));
}
// (b) localDb.ts deve conter SOMENTE import/export + comentários (sem lógica).
// Remove comentários e strings, depois procura declarações de runtime.
export function hasLogic(localDbSource) {
const stripped = localDbSource
// comentários de bloco
.replace(/\/\*[\s\S]*?\*\//g, "")
// comentários de linha
.replace(/\/\/[^\n]*/g, "")
// template strings
.replace(/`(?:\\[\s\S]|[^\\`])*`/g, '""')
// strings simples/duplas (paths de import etc.)
.replace(/"(?:\\.|[^"\\])*"/g, '""')
.replace(/'(?:\\.|[^'\\])*'/g, '""');
// function/class declaradas, ou atribuição a função (const X = (…) =>, const X = function).
const logicPatterns = [
/(^|[^.\w])function\s+[A-Za-z_$]/, // function decl (não method .foo())
/(^|[^.\w])class\s+[A-Za-z_$]/, // class decl
/(?:const|let|var)\s+[A-Za-z_$][\w$]*\s*=\s*(?:async\s*)?\(/, // const X = (…) ... (arrow/call)
/(?:const|let|var)\s+[A-Za-z_$][\w$]*\s*=\s*(?:async\s+)?function\b/, // const X = function
];
return logicPatterns.some((rx) => rx.test(stripped));
}
// SQL cru é sempre uma STRING passada a db.prepare()/exec(): casamos os padrões
// SÓ dentro de literais de string (não em código JS — `import … from`, `.set(`,
// `new Set(`, `delete x` etc. são falsos positivos se varrermos o código todo).
const SQL_PATTERNS = [
/\bSELECT\b[\s\S]*?\bFROM\b/i, // SELECT … FROM (multi-linha)
/\bINSERT\s+INTO\b/i,
/\bUPDATE\b[\s\S]*?\bSET\b/i, // UPDATE … SET (multi-linha)
/\bDELETE\s+FROM\b/i,
/\bCREATE\s+TABLE\b/i,
];
// Remove comentários (linha // … e blocos /* */) — SQL em comentário não conta.
function stripComments(source) {
return source.replace(/\/\*[\s\S]*?\*\//g, "").replace(/\/\/[^\n]*/g, "");
}
// Extrai o conteúdo de todos os literais de string (template, aspas duplas, aspas
// simples) de um trecho de código já sem comentários. Retorna a concatenação dos
// corpos — é nesse corpo que SQL cru vive.
export function extractStringLiterals(code) {
const re = /`(?:\\[\s\S]|[^\\`])*`|"(?:\\.|[^"\\])*"|'(?:\\.|[^'\\])*'/g;
const out = [];
let m;
while ((m = re.exec(code))) {
// tira as aspas/crases delimitadoras
out.push(m[0].slice(1, -1));
}
return out.join("\n\n"); // separador que nenhum padrão SQL atravessa
}
// (c) Arquivos com SQL cru dentro de literais de string (linhas não-comentário),
// fora do allowlist.
export function findRawSql(files, allowlist = KNOWN_RAW_SQL) {
const offenders = [];
for (const file of files) {
const rel = path.relative(cwd, file).replace(/\\/g, "/");
if (allowlist.has(rel)) continue;
let src;
try {
src = fs.readFileSync(file, "utf8");
} catch {
continue;
}
const literals = extractStringLiterals(stripComments(src));
if (SQL_PATTERNS.some((rx) => rx.test(literals))) {
offenders.push(rel);
}
}
return offenders;
}
// Coleta os arquivos sujeitos à checagem (c): rotas de API + handlers de stream.
export function collectSqlScanFiles(apiDir = API_DIR, handlersDir = HANDLERS_DIR) {
const routes = walk(apiDir).filter((p) => /(^|\/)route\.tsx?$/.test(p.replace(/\\/g, "/")));
const handlers = fs.existsSync(handlersDir)
? fs
.readdirSync(handlersDir, { withFileTypes: true })
.filter((e) => e.isFile() && /\.tsx?$/.test(e.name))
.map((e) => path.join(handlersDir, e.name))
: [];
return [...routes, ...handlers];
}
function main() {
const failures = [];
// (a) re-export completeness
const dbModules = collectDbModules();
const reexported = extractReexportedModules(fs.readFileSync(LOCAL_DB, "utf8"));
const missing = findMissingReexports(dbModules, reexported);
if (missing.length) {
failures.push(
`[#2 re-export] ${missing.length} módulo(s) db/ não re-exportado(s) por src/lib/localDb.ts:\n` +
missing.map((m) => ` ✗ src/lib/db/${m}.ts`).join("\n") +
`\n → re-exporte de src/lib/localDb.ts (apenas a lista de re-export, nada de lógica)` +
` ou adicione a KNOWN_UNEXPORTED com justificativa (import direto de "@/lib/db/${missing[0]}").`
);
}
// (b) localDb sem lógica
if (hasLogic(fs.readFileSync(LOCAL_DB, "utf8"))) {
failures.push(
`[#2 sem-lógica] src/lib/localDb.ts contém lógica (function/class/arrow). É camada de` +
` re-export apenas — mova a lógica para um módulo src/lib/db/.`
);
}
// (c) SQL cru fora de db/
const rawSql = findRawSql(collectSqlScanFiles());
if (rawSql.length) {
failures.push(
`[#5 sql-cru] ${rawSql.length} arquivo(s) com SQL cru fora de src/lib/db/:\n` +
rawSql.map((f) => `${f}`).join("\n") +
`\n → mova o SQL para um módulo src/lib/db/ (nunca SQL cru em rota/handler)` +
` ou congele em KNOWN_RAW_SQL com justificativa.`
);
}
if (failures.length) {
console.error(`[check-db-rules] FALHOU:\n\n` + failures.join("\n\n"));
process.exit(1);
}
console.log(
`[check-db-rules] OK (${dbModules.length} módulos db/, ${reexported.size} re-exportados, ` +
`${KNOWN_UNEXPORTED.size} congelados; ${KNOWN_RAW_SQL.size} ofensores de SQL congelados)`
);
}
if (import.meta.url === pathToFileURL(process.argv[1] || "").href) main();