feat(homolog): real-environment E2E homologation suite (npm run homolog) (#7133)

* feat(homolog): scaffolding da suíte de homologação E2E (deps + npm run homolog) * feat(homolog): L0 avaliador de paridade de deploy (TDD) * feat(homolog): L1a ciclo de vida de API key efêmera (login admin -> create -> revoke) * feat(homolog): L1b suite httpYac de API (models, chat, auth de management, health) * feat(homolog): L1c checker SSE de streaming real (TDD no parser) * feat(homolog): L2 smoke de providers reais via promptfoo gerado do catálogo * feat(homolog): L4a Playwright homolog config + login storageState * feat(homolog): L4b smoke de todas as rotas do dashboard (descoberta via fs) * feat(homolog): L4c fluxo criar/revogar API key pela UI * fix(homolog): resiliencia real-environment — stream:false no smoke promptfoo, retry de socket keep-alive, key efemera com sufixo unico * feat(homolog): L5 orquestrador npm run homolog + relatorio CTRF unificado * docs(homolog): guia de operacao da suite + fragment de changelog + allowlist env-doc-sync * fix(homolog): paraleliza o sweep de rotas do dashboard (fullyParallel + 8 workers) * fix(homolog): isola outputs crus em homolog-report/raw para nao quebrar o ctrf merge * fix(homolog): outputDir absoluto do reporter CTRF da UI (path relativo escapava do worktree) * chore(quality): allowlist the 5 homolog-suite devDependencies (ctrf-io trio, httpyac, promptfoo) after registry verification * chore(quality): register the homolog Playwright suite as a test-discovery collector (run.mjs -> tests/homolog/ui)
2026-07-26 09:52:11 +03:00 · 2026-07-14 16:24:11 -03:00
parent a96e4b58f8
commit c97d2a6ae2
26 changed files with 10412 additions and 250 deletions
--- a/.env.homolog.example
+++ b/.env.homolog.example
@@ -0,0 +1,9 @@
+# Homologação E2E real — copie para .env.homolog (NUNCA commitar o real)
+HOMOLOG_BASE_URL=http://192.168.0.15:20128
+# Senha de management do dashboard da VPS (a mesma do /login)
+HOMOLOG_ADMIN_PASSWORD=
+# Deixe vazio: a suíte cria uma API key efêmera via admin e revoga no fim.
+# Só preencha para depurar uma camada isolada com uma key fixa.
+HOMOLOG_API_KEY=
+# Tier crítico (chat real, max_tokens=5). Demais providers: só validação de catálogo.
+HOMOLOG_CRITICAL_PROVIDERS=openai,anthropic,gemini,codex,grok,glm,deepseek,openrouter
--- a/.gitignore
+++ b/.gitignore
@@ -72,6 +72,7 @@ yarn-error.log*
 # env files (can opt-in for committing if needed)
 .env*
 !.env.example
+!.env.homolog.example
 # Provider API keys (never commit)
 *.api-key
 .nvidia-api-key
@@ -242,3 +243,9 @@ _artifacts/

 # CI/local quality artifacts (eslint-results.json, etc.)
 .artifacts/
+
+# Homologation E2E suite (npm run homolog) — real-environment credentials + report output
+.env.homolog
+tests/homolog/.auth/
+tests/homolog/ui/.auth/
+homolog-report/
--- a/changelog.d/features/homolog-e2e-suite.md
+++ b/changelog.d/features/homolog-e2e-suite.md
@@ -0,0 +1 @@
+- **Homologation suite**: new `npm run homolog` runs the full release-homologation battery against the deployed VPS — health/version parity, API + real SSE streaming with an ephemeral API key (created and revoked by the run), minimal-cost real-provider smoke (promptfoo generated from the live catalog), and a Playwright sweep that loads every dashboard route and exercises the API-key UI flow — emitting a unified CTRF report that backs the release STOP #2 checklist
--- a/config/quality/dependency-allowlist.json
+++ b/config/quality/dependency-allowlist.json
@@ -45,6 +45,7 @@
    "concurrently",
    "cross-env",
    "csv-stringify",
+    "ctrf",
    "dompurify",
    "dpdm",
    "electron",
@@ -63,6 +64,7 @@
    "glob",
    "http-proxy-middleware",
    "https-proxy-agent",
+    "httpyac",
    "husky",
    "ink",
    "ink-spinner",
@@ -74,6 +76,7 @@
    "jscpd",
    "jsdom",
    "jsonc-parser",
+    "junit-to-ctrf",
    "keytar",
    "knip",
    "license-checker-rseidelsohn",
@@ -99,7 +102,9 @@
    "pino-abstract-transport",
    "pino-pretty",
    "playwright",
+    "playwright-ctrf-json-reporter",
    "prettier",
+    "promptfoo",
    "react",
    "react-dom",
    "react-is",
--- a/docs/ops/HOMOLOGATION.md
+++ b/docs/ops/HOMOLOGATION.md
@@ -0,0 +1,104 @@
+---
+title: "Homologation Suite (npm run homolog)"
+version: 3.8.49
+lastUpdated: 2026-07-14
+---
+
+# Homologation Suite (`npm run homolog`)
+
+Real-environment E2E validation of the OmniRoute deploy running on the homologation VPS
+(`HOMOLOG_BASE_URL`, e.g. `http://192.168.0.15:20128`). One command replaces the manual
+release STOP #2 checklist with an automated, evidence-producing run.
+
+## What it covers
+
+| Layer                  | What it checks                                                                                                                                                                                            | Implementation                                                                |
+| ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------- |
+| L0 — health/parity     | `/api/monitoring/health` responds `200` with `status: "healthy"` and the expected version                                                                                                                 | `scripts/homolog/lib/parity.mjs`                                              |
+| L1a — ephemeral key    | Admin login → `POST /api/keys` creates a scoped API key for the run, revoked (`DELETE /api/keys/:id`) in a `finally` block regardless of outcome                                                          | `scripts/homolog/lib/adminClient.mjs`                                         |
+| L1b — API surface      | `/v1/models` catalog, a real non-streaming chat completion (tier-critical model, `max_tokens: 5`), an invalid-key `401`, and public `/api/monitoring/health`                                              | `tests/homolog/api/core.http` (httpYac)                                       |
+| L1c — SSE streaming    | Real streaming chat completion; asserts `text/event-stream`, at least one content delta, and a `[DONE]` terminator                                                                                        | `scripts/homolog/lib/sseCheck.mjs`                                            |
+| L2 — real providers    | One minimal-cost chat request per critical provider present in the live `/v1/models` catalog, generated on the fly via promptfoo                                                                          | `scripts/homolog/gen-promptfoo.mjs` + `scripts/homolog/lib/providerTiers.mjs` |
+| L4a — UI auth          | Logs in once via the real login form and reuses the session (`storageState`) across the UI layer                                                                                                          | `tests/homolog/ui/auth.setup.ts`                                              |
+| L4b — UI routes        | Every static `page.tsx` under `src/app/(dashboard)/dashboard` (discovered from the filesystem, dynamic `[param]` routes skipped) loads without an HTTP error, a page error, or the Next.js error boundary | `tests/homolog/ui/routes.spec.ts`                                             |
+| L4c — UI critical flow | Creates an API key through the dashboard UI and revokes it again (leaves no residue on the VPS)                                                                                                           | `tests/homolog/ui/api-key-flow.spec.ts`                                       |
+| L5 — unified report    | Merges httpYac (via `junit-to-ctrf`), the promptfoo→CTRF adapter, and the Playwright CTRF reporter into one `homolog-ctrf.json`, plus a human-readable `homolog-report/summary.md`                        | `scripts/homolog/run.mjs`                                                     |
+
+Zero LLM involvement in the replay itself — this is a deterministic regression battery,
+not an eval. AI only enters in future maintenance work (see Roadmap below).
+
+## Prerequisites
+
+1. Copy `.env.homolog.example` to `.env.homolog` (gitignored — never commit it) and fill in:
+   - `HOMOLOG_BASE_URL` — the target deploy, e.g. `http://192.168.0.15:20128`.
+   - `HOMOLOG_ADMIN_PASSWORD` — the dashboard management password for that deploy.
+   - `HOMOLOG_CRITICAL_PROVIDERS` — comma-separated provider prefixes that get a real
+     smoke chat request (e.g. `openai,anthropic,gemini,codex,grok,glm,deepseek,openrouter`).
+   - `HOMOLOG_API_KEY` — leave empty in normal runs; the suite creates and revokes its
+     own ephemeral key. Only set this to debug a single layer in isolation.
+2. `npm install` in the repo (the suite's dependencies — `httpyac`, `promptfoo`,
+   `playwright-ctrf-json-reporter`, `junit-to-ctrf`, `ctrf` — are regular devDependencies).
+3. `npx playwright install` if the browser binaries are not already present.
+
+## How to run
+
+```bash
+npm run homolog
+```
+
+To validate against a deploy whose version does not match the local `package.json`
+(e.g. a homologation box still on a previous patch release), override the expected
+version explicitly:
+
+```bash
+HOMOLOG_EXPECT_VERSION=3.8.47 npm run homolog
+```
+
+The run exits non-zero if any layer fails, and always attempts to revoke the ephemeral
+API key it created, even on failure (`finally` block in `scripts/homolog/run.mjs`).
+
+## Reading the report
+
+All output lands in `homolog-report/` (gitignored):
+
+- `summary.md` — the same table printed to stdout, one row per layer (✅/❌ + detail).
+- `homolog-ctrf.json` — the unified CTRF report (merge of API/SSE, provider-smoke, and
+  UI results) — this is the artifact to attach to a release STOP #2 checklist.
+- `httpyac-junit.xml`, `api-ctrf.json`, `providers-ctrf.json`, `ui-ctrf.json` — the
+  per-layer raw/intermediate reports.
+- `promptfooconfig.yaml`, `provider-misses.json` — the generated promptfoo config for
+  the current run and any critical providers that were missing from the live catalog.
+
+A failing L0 aborts immediately (no ephemeral key is created) since a version/health
+mismatch means every downstream layer would be validating the wrong deploy.
+
+## Re-baselining when the UI changes legitimately
+
+L4b (route smoke) and L4c (API-key UI flow) are driven by real DOM locators, not
+snapshots, so most legitimate UI changes do not require any suite update. When a change
+does break a locator (e.g. a renamed button label or a moved settings page):
+
+1. Re-confirm the locator against the current source (the specs already document which
+   file/line each locator was confirmed against — follow the same pattern, don't guess).
+2. Update the spec in `tests/homolog/ui/`.
+3. Re-run `npm run homolog` (or just the affected Playwright spec) against the VPS to
+   confirm the fix, then commit.
+
+There is no visual/pixel baseline in this suite (F1) — see Roadmap for that.
+
+## Roadmap (F2 / F3)
+
+Design and phased rollout live in the internal planning spec
+`_tasks/superpowers/specs/2026-07-13-homolog-e2e-suite-design.md` (not linked — internal
+`_tasks/` artifact, not part of this repo's tracked docs). Summary:
+
+- **F2** — full walkthrough recording → Playwright Test Agents (`planner`/`generator`)
+  turn it into flow specs (create combo, test provider, edit settings, MCP tools) +
+  visual regression baseline (Lost Pixel) with masks over dynamic data (metrics,
+  timestamps, logs) + a `healer` maintenance routine per release.
+- **F3** — resilience/contract/wiring coverage: toxiproxy + a fake OpenAI-compatible
+  provider on the devbox, a `homolog-resilience` combo on the VPS pointed at it
+  (injected timeout → assert fallback + circuit breaker open/close via
+  `/api/monitoring/health`); gated Schemathesis contract testing against
+  `docs/openapi.yaml` (low `--max-examples`, fixed seeds, non-LLM endpoints only); and
+  wiring `npm run homolog` + its `summary.md` into the `/generate-release` STOP #2 phase.
--- a/package-lock.json
+++ b/package-lock.json
--- a/package.json
+++ b/package.json
@@ -87,6 +87,7 @@
    "build:release": "rm -rf .build dist && OMNIROUTE_BUILD_SHA=$(git rev-parse --short HEAD) npm run build && npm run build:cli && node scripts/build/write-build-sha.mjs",
    "build:native:tproxy": "cd src/mitm/tproxy/native && npx --yes node-gyp rebuild",
    "start": "node scripts/dev/run-next.mjs start",
+    "homolog": "node scripts/homolog/run.mjs",
    "lint": "eslint . --cache --cache-location .eslintcache --suppressions-location config/quality/eslint-suppressions.json",
    "lint:json": "node scripts/quality/run-eslint-json.mjs",
    "lint:md": "npx --yes markdownlint-cli2 \"docs/**/*.md\" \"*.md\" \"!docs/i18n\" \"!docs/research\"",
@@ -329,21 +330,26 @@
    "c8": "^11.0.0",
    "concurrently": "^10.0.3",
    "cross-env": "^10.1.0",
+    "ctrf": "^0.2.1",
    "dpdm": "^4.2.0",
    "eslint": "^9.39.4",
    "eslint-config-next": "16.2.10",
    "eslint-plugin-sonarjs": "^4.1.0",
    "fast-check": "^4.8.0",
    "glob": "^13.0.6",
+    "httpyac": "^6.16.7",
    "husky": "^9.1.7",
    "jscpd": "^4.2.5",
    "jsdom": "^29.1.1",
+    "junit-to-ctrf": "^0.0.14",
    "knip": "^6.18.0",
    "license-checker-rseidelsohn": "^5.0.1",
    "lint-staged": "^17.0.8",
    "lockfile-lint": "^5.0.0",
    "node-loader": "^2.1.0",
+    "playwright-ctrf-json-reporter": "^0.0.29",
    "prettier": "^3.8.3",
+    "promptfoo": "^0.121.18",
    "size-limit": "^12.1.0",
    "tailwindcss": "^4.3.0",
    "type-coverage": "^2.29.7",
--- a/scripts/check/check-env-doc-sync.mjs
+++ b/scripts/check/check-env-doc-sync.mjs
@@ -101,6 +101,15 @@ const IGNORE_FROM_CODE = new Set([
  // ("http://192.168.0.15:20128" / null), never OmniRoute runtime config (#5151).
  "COMBO_LIVE_BASE_URL",
  "COMBO_LIVE_API_KEY",
+  // Homologation E2E suite (npm run homolog) vars — configured via the dedicated
+  // .env.homolog file (template: .env.homolog.example), never in the runtime .env.
+  // Test/ops-only signals against the homologation VPS, same class as COMBO_LIVE_*.
+  // See docs/ops/HOMOLOGATION.md.
+  "HOMOLOG_BASE_URL",
+  "HOMOLOG_ADMIN_PASSWORD",
+  "HOMOLOG_API_KEY",
+  "HOMOLOG_CRITICAL_PROVIDERS",
+  "HOMOLOG_EXPECT_VERSION",
  // update-notifier opt-out for the CLI binary.
  "OMNIROUTE_NO_UPDATE_NOTIFIER",
  // Headless CLI execution flag for Electron.
--- a/scripts/check/check-test-discovery.mjs
+++ b/scripts/check/check-test-discovery.mjs
@@ -127,6 +127,13 @@ export const COLLECTORS = [
    glob: "tests/e2e/protocol-clients.test.ts",
    sources: ["scripts/dev/run-protocol-clients-tests.mjs"],
  },
+  // Playwright — suíte de homologação real (npm run homolog, L4 UI): run.mjs invoca
+  // `playwright test -c tests/homolog/ui/playwright.config.ts` (testMatch **/*.spec.ts).
+  {
+    glob: "tests/homolog/ui/*.spec.ts",
+    sources: ["scripts/homolog/run.mjs"],
+    anchors: { "scripts/homolog/run.mjs": "tests/homolog/ui/playwright.config.ts" },
+  },
 ];

 const escapeRe = (s) => s.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
--- a/scripts/homolog/gen-promptfoo.mjs
+++ b/scripts/homolog/gen-promptfoo.mjs
@@ -0,0 +1,52 @@
+import fs from "node:fs";
+import path from "node:path";
+import { pickSmokeModels } from "./lib/providerTiers.mjs";
+
+const baseUrl = process.env.HOMOLOG_BASE_URL;
+const critical = (process.env.HOMOLOG_CRITICAL_PROVIDERS || "").split(",").filter(Boolean);
+
+const res = await fetch(`${baseUrl}/v1/models`, {
+  headers: { Authorization: `Bearer ${process.env.HOMOLOG_API_KEY}` },
+});
+if (!res.ok) throw new Error(`/v1/models HTTP ${res.status}`);
+const catalog = (await res.json()).data;
+
+const picks = pickSmokeModels(catalog, critical);
+const missing = picks.filter((p) => !p.model);
+const providers = picks
+  .filter((p) => p.model)
+  .map((p) => ({
+    id: `openai:chat:${p.model}`,
+    label: p.provider,
+    config: {
+      apiBaseUrl: `${baseUrl}/v1`,
+      apiKeyEnvar: "HOMOLOG_API_KEY",
+      max_tokens: 5,
+      temperature: 0,
+      // OmniRoute streama por default quando "stream" é omitido (streamDefaultMode
+      // legacy) — o parser JSON do promptfoo precisa da resposta non-stream.
+      passthrough: { stream: false, max_tokens: 5 },
+    },
+  }));
+
+const config = {
+  description: "OmniRoute homolog — smoke real 1 request/provider crítico",
+  prompts: ["Reply with exactly: OK"],
+  providers,
+  // O smoke valida o WIRING do provider (respondeu sem erro), não o comportamento
+  // do modelo: com max_tokens=5, modelos de reasoning podem gastar o budget antes
+  // de emitir o "OK" literal — icontains seria falso-positivo de quebra.
+  tests: [{ assert: [{ type: "javascript", value: "typeof output === 'string'" }] }],
+};
+fs.mkdirSync("homolog-report/raw", { recursive: true });
+fs.writeFileSync(
+  path.join("homolog-report", "promptfooconfig.yaml"),
+  JSON.stringify(config, null, 2) // promptfoo aceita JSON como config YAML-compatível
+);
+fs.writeFileSync(
+  path.join("homolog-report", "raw", "provider-misses.json"),
+  JSON.stringify(missing, null, 2)
+);
+console.log(
+  `[gen-promptfoo] ${providers.length} providers no smoke, ${missing.length} misses de catálogo`
+);
--- a/scripts/homolog/lib/adminClient.mjs
+++ b/scripts/homolog/lib/adminClient.mjs
@@ -0,0 +1,65 @@
+// Cookie confirmado em src/app/api/auth/login/route.ts (cookieStore.set("auth_token", ...))
+// e em src/shared/utils/apiAuth.ts (isDashboardSessionAuthenticated lê "auth_token").
+const TOKEN_COOKIE = "auth_token";
+
+export function extractJwtCookie(setCookies) {
+  for (const c of setCookies || []) {
+    const m = c.match(new RegExp(`^(${TOKEN_COOKIE}=[^;]+)`));
+    if (m) return m[1];
+  }
+  return null;
+}
+
+export function extractApiKey(body) {
+  if (!body?.key || !body?.id) throw new Error("POST /api/keys sem key/id no corpo");
+  return { key: body.key, id: body.id };
+}
+
+// fetch com 1 retry para erros de socket (keep-alive reciclado pelo servidor
+// entre requests espaçados derruba o 1º write com EPIPE/other side closed).
+async function fetchRetry(url, init, retries = 1) {
+  try {
+    return await fetch(url, init);
+  } catch (err) {
+    if (retries > 0) {
+      await new Promise((r) => setTimeout(r, 1_000));
+      return fetchRetry(url, init, retries - 1);
+    }
+    throw err;
+  }
+}
+
+/** Login admin → cria API key efêmera. Retorna {key, id, cookie, revoke()}. */
+export async function createEphemeralKey(baseUrl, password) {
+  const login = await fetchRetry(`${baseUrl}/api/auth/login`, {
+    method: "POST",
+    headers: { "Content-Type": "application/json" },
+    body: JSON.stringify({ password }),
+  });
+  if (!login.ok) throw new Error(`login falhou: HTTP ${login.status}`);
+  const cookie = extractJwtCookie(login.headers.getSetCookie());
+  if (!cookie) throw new Error("login sem cookie de sessão");
+
+  // sufixo único por run: dois runs paralelos (ou um cleanup por nome) nunca colidem
+  const name = `homolog-${new Date().toISOString().slice(0, 10)}-${Math.random().toString(36).slice(2, 8)}`;
+  const create = await fetchRetry(`${baseUrl}/api/keys`, {
+    method: "POST",
+    headers: { "Content-Type": "application/json", cookie },
+    body: JSON.stringify({ name }),
+  });
+  if (!create.ok) throw new Error(`criação de key falhou: HTTP ${create.status}`);
+  const { key, id } = extractApiKey(await create.json());
+
+  return {
+    key,
+    id,
+    cookie,
+    async revoke() {
+      const del = await fetchRetry(`${baseUrl}/api/keys/${id}`, {
+        method: "DELETE",
+        headers: { cookie },
+      });
+      if (!del.ok) throw new Error(`revogação da key ${id} falhou: HTTP ${del.status}`);
+    },
+  };
+}
--- a/scripts/homolog/lib/parity.mjs
+++ b/scripts/homolog/lib/parity.mjs
@@ -0,0 +1,15 @@
+/**
+ * Avaliação pura de paridade do deploy (testável sem rede).
+ * @param {{status?: string, version?: string}} health corpo de /api/monitoring/health
+ * @param {{expectedVersion: string, httpStatus: number}} ctx
+ * @returns {{ok: boolean, failures: string[]}}
+ */
+export function evaluateParity(health, ctx) {
+  const failures = [];
+  if (ctx.httpStatus !== 200) failures.push(`health HTTP ${ctx.httpStatus} (esperado 200)`);
+  if (health?.status !== "healthy")
+    failures.push(`status "${health?.status}" (esperado "healthy")`);
+  if (health?.version !== ctx.expectedVersion)
+    failures.push(`version "${health?.version}" (esperado "${ctx.expectedVersion}")`);
+  return { ok: failures.length === 0, failures };
+}
--- a/scripts/homolog/lib/promptfooToCtrf.mjs
+++ b/scripts/homolog/lib/promptfooToCtrf.mjs
@@ -0,0 +1,24 @@
+export function promptfooToCtrf(output) {
+  const rows = output?.results?.results || [];
+  const tests = rows.map((r) => ({
+    name: `provider-smoke: ${r.provider?.label || r.provider?.id || "?"}`,
+    status: r.success ? "passed" : "failed",
+    duration: Math.round(r.latencyMs || 0),
+    ...(r.error ? { message: String(r.error).slice(0, 300) } : {}),
+  }));
+  const passed = tests.filter((t) => t.status === "passed").length;
+  return {
+    results: {
+      tool: { name: "promptfoo" },
+      summary: {
+        tests: tests.length,
+        passed,
+        failed: tests.length - passed,
+        pending: 0,
+        skipped: 0,
+        other: 0,
+      },
+      tests,
+    },
+  };
+}
--- a/scripts/homolog/lib/providerTiers.mjs
+++ b/scripts/homolog/lib/providerTiers.mjs
@@ -0,0 +1,7 @@
+/** Escolhe 1 modelo por provider crítico a partir do catálogo /v1/models. */
+export function pickSmokeModels(catalog, criticalProviders) {
+  return criticalProviders.map((provider) => {
+    const hit = catalog.find((m) => m.id.startsWith(`${provider}/`));
+    return { provider, model: hit ? hit.id : null };
+  });
+}
--- a/scripts/homolog/lib/sseCheck.mjs
+++ b/scripts/homolog/lib/sseCheck.mjs
@@ -0,0 +1,80 @@
+export function parseSseChunk(text) {
+  // Itera LINHAS dentro de cada bloco: a VPS emite comment-lines SSE
+  // (": x-omniroute-*") no mesmo bloco do "data: [DONE]", então olhar só o
+  // início do bloco perde o terminador.
+  const events = [];
+  for (const block of text.split(/\n\n/)) {
+    for (const line of block.split("\n")) {
+      const t = line.trim();
+      if (t.startsWith("data:")) events.push(t.slice(5).trim());
+    }
+  }
+  return events;
+}
+
+export function summarizeStream(events) {
+  let contentDeltas = 0;
+  let done = false;
+  for (const e of events) {
+    if (e === "[DONE]") {
+      done = true;
+      continue;
+    }
+    try {
+      const j = JSON.parse(e);
+      if (j.choices?.[0]?.delta?.content) contentDeltas++;
+    } catch {
+      /* fragmento parcial — ignorado; o caller acumula buffer */
+    }
+  }
+  const ok = contentDeltas >= 1 && done;
+  return { ok, contentDeltas, done };
+}
+
+/** Faz 1 chat streaming real e valida o protocolo SSE ponta-a-ponta. */
+export async function checkSse(baseUrl, apiKey, model, { retries = 1 } = {}) {
+  try {
+    const res = await fetch(`${baseUrl}/v1/chat/completions`, {
+      method: "POST",
+      headers: { Authorization: `Bearer ${apiKey}`, "Content-Type": "application/json" },
+      body: JSON.stringify({
+        model,
+        messages: [{ role: "user", content: "Reply with exactly: OK" }],
+        max_tokens: 5,
+        stream: true,
+      }),
+    });
+    if (res.status !== 200) return { ok: false, failures: [`HTTP ${res.status}`] };
+    const ct = res.headers.get("content-type") || "";
+    if (!ct.includes("text/event-stream")) return { ok: false, failures: [`content-type "${ct}"`] };
+
+    const events = [];
+    let buffer = "";
+    const reader = res.body.getReader();
+    const decoder = new TextDecoder();
+    const deadline = Date.now() + 60_000;
+    while (Date.now() < deadline) {
+      const { done, value } = await reader.read();
+      if (done) break;
+      buffer += decoder.decode(value, { stream: true });
+      const lastSep = buffer.lastIndexOf("\n\n");
+      if (lastSep >= 0) {
+        events.push(...parseSseChunk(buffer.slice(0, lastSep + 2)));
+        buffer = buffer.slice(lastSep + 2);
+      }
+    }
+    // flush do resto do buffer (último bloco pode chegar sem "\n\n" no read final)
+    if (buffer.trim()) events.push(...parseSseChunk(buffer));
+    const s = summarizeStream(events);
+    return { ok: s.ok, failures: s.ok ? [] : [`contentDeltas=${s.contentDeltas} done=${s.done}`] };
+  } catch (err) {
+    // Socket keep-alive reciclado pelo servidor entre requests é transitório —
+    // 1 retry antes de reportar falha. Erro persistente é FALHA da camada,
+    // nunca crash do orquestrador.
+    if (retries > 0) {
+      await new Promise((r) => setTimeout(r, 1_000));
+      return checkSse(baseUrl, apiKey, model, { retries: retries - 1 });
+    }
+    return { ok: false, failures: [`fetch/stream error: ${err?.cause?.message || err.message}`] };
+  }
+}
--- a/scripts/homolog/run.mjs
+++ b/scripts/homolog/run.mjs
@@ -0,0 +1,169 @@
+#!/usr/bin/env node
+import { execSync, spawnSync } from "node:child_process";
+import fs from "node:fs";
+import { evaluateParity } from "./lib/parity.mjs";
+import { createEphemeralKey } from "./lib/adminClient.mjs";
+import { checkSse } from "./lib/sseCheck.mjs";
+import { promptfooToCtrf } from "./lib/promptfooToCtrf.mjs";
+
+// ── env ──────────────────────────────────────────────────────────────────
+if (fs.existsSync(".env.homolog")) {
+  for (const line of fs.readFileSync(".env.homolog", "utf8").split("\n")) {
+    const m = line.match(/^([A-Z_]+)=(.*)$/);
+    if (m && !process.env[m[1]]) process.env[m[1]] = m[2];
+  }
+}
+const BASE = process.env.HOMOLOG_BASE_URL;
+if (!BASE || !process.env.HOMOLOG_ADMIN_PASSWORD) {
+  console.error("Configure .env.homolog (HOMOLOG_BASE_URL, HOMOLOG_ADMIN_PASSWORD)");
+  process.exit(2);
+}
+fs.rmSync("homolog-report", { recursive: true, force: true });
+// raw/ fica FORA do merge CTRF: `ctrf merge` tenta mesclar qualquer *.json com
+// chave "results" e quebra no output cru do promptfoo.
+fs.mkdirSync("homolog-report/raw", { recursive: true });
+const layers = []; // {name, ok, detail}
+const record = (name, ok, detail = "") => {
+  layers.push({ name, ok, detail });
+  console.log(`${ok ? "✅" : "❌"} ${name}${detail ? ` — ${detail}` : ""}`);
+};
+
+// ── L0 saúde/paridade ────────────────────────────────────────────────────
+const expectedVersion =
+  process.env.HOMOLOG_EXPECT_VERSION || JSON.parse(fs.readFileSync("package.json", "utf8")).version;
+const healthRes = await fetch(`${BASE}/api/monitoring/health`);
+const health = await healthRes.json().catch(() => ({}));
+const parity = evaluateParity(health, { expectedVersion, httpStatus: healthRes.status });
+record("L0 saúde/paridade", parity.ok, parity.failures.join("; "));
+if (!parity.ok) {
+  console.error("Deploy divergente — abortando.");
+  writeSummary(layers, BASE, expectedVersion);
+  process.exit(1);
+}
+
+// ── chave efêmera ────────────────────────────────────────────────────────
+const eph = await createEphemeralKey(BASE, process.env.HOMOLOG_ADMIN_PASSWORD);
+process.env.HOMOLOG_API_KEY = eph.key;
+try {
+  // modelo de smoke = 1º do tier crítico presente no catálogo
+  const models = (
+    await (
+      await fetch(`${BASE}/v1/models`, { headers: { Authorization: `Bearer ${eph.key}` } })
+    ).json()
+  ).data;
+  const critical = (process.env.HOMOLOG_CRITICAL_PROVIDERS || "openai").split(",");
+  const smokeModel =
+    models.find((m) => critical.some((p) => m.id.startsWith(`${p}/`)))?.id || models[0].id;
+
+  // ── L1 httpYac + SSE ───────────────────────────────────────────────────
+  const hy = spawnSync(
+    "npx",
+    [
+      "httpyac",
+      "send",
+      "tests/homolog/api/core.http",
+      "--all",
+      "--var",
+      `baseUrl=${BASE}`,
+      "--var",
+      `apiKey=${eph.key}`,
+      "--var",
+      `smokeModel=${smokeModel}`,
+      "--junit",
+      "--output",
+      "none",
+    ],
+    { encoding: "utf8" }
+  );
+  fs.writeFileSync("homolog-report/httpyac-junit.xml", hy.stdout || "");
+  record("L1 API (httpYac)", hy.status === 0);
+  const sse = await checkSse(BASE, eph.key, smokeModel);
+  record("L1 SSE streaming", sse.ok, (sse.failures || []).join("; "));
+
+  // ── L2 providers reais ─────────────────────────────────────────────────
+  try {
+    execSync("node scripts/homolog/gen-promptfoo.mjs", { stdio: "inherit", env: process.env });
+    spawnSync(
+      "npx",
+      [
+        "promptfoo",
+        "eval",
+        "-c",
+        "homolog-report/promptfooconfig.yaml",
+        "-o",
+        "homolog-report/raw/promptfoo.json",
+        "--no-cache",
+      ],
+      { encoding: "utf8", env: process.env }
+    );
+    const pfOut = JSON.parse(fs.readFileSync("homolog-report/raw/promptfoo.json", "utf8"));
+    const pfCtrf = promptfooToCtrf(pfOut);
+    fs.writeFileSync("homolog-report/providers-ctrf.json", JSON.stringify(pfCtrf, null, 2));
+    record(
+      "L2 providers reais",
+      pfCtrf.results.summary.failed === 0,
+      `${pfCtrf.results.summary.passed}/${pfCtrf.results.summary.tests} providers OK`
+    );
+  } catch (err) {
+    // gerador/eval quebrando é falha da camada — o run continua para o L4 e o cleanup
+    record("L2 providers reais", false, err.message);
+  }
+
+  // ── L4 UI ──────────────────────────────────────────────────────────────
+  const pw = spawnSync(
+    "npx",
+    ["playwright", "test", "-c", "tests/homolog/ui/playwright.config.ts"],
+    {
+      stdio: "inherit",
+      env: process.env,
+    }
+  );
+  record("L4 UI (Playwright)", pw.status === 0);
+} finally {
+  await eph
+    .revoke()
+    .then(() => record("cleanup: key efêmera revogada", true))
+    .catch((e) => record("cleanup: key efêmera revogada", false, e.message));
+}
+
+// ── L5 relatório unificado ───────────────────────────────────────────────
+spawnSync(
+  "npx",
+  ["junit-to-ctrf", "homolog-report/httpyac-junit.xml", "-o", "homolog-report/api-ctrf.json"],
+  {
+    stdio: "inherit",
+  }
+);
+spawnSync(
+  "npx",
+  [
+    "ctrf",
+    "merge",
+    "homolog-report",
+    "--output",
+    "homolog-ctrf.json",
+    "--output-dir",
+    "homolog-report",
+  ],
+  {
+    stdio: "inherit",
+  }
+);
+
+writeSummary(layers, BASE, expectedVersion);
+const failed = layers.filter((l) => !l.ok);
+process.exit(failed.length ? 1 : 0);
+
+function writeSummary(rows, base, version) {
+  const md = [
+    "# Homologação — relatório",
+    "",
+    `Alvo: ${base} · versão esperada: ${version}`,
+    "",
+    "| camada | resultado | detalhe |",
+    "|---|---|---|",
+    ...rows.map((l) => `| ${l.name} | ${l.ok ? "✅" : "❌"} | ${l.detail} |`),
+  ].join("\n");
+  fs.writeFileSync("homolog-report/summary.md", md);
+  console.log(`\n${md}\n\nRelatório: homolog-report/ (CTRF unificado: homolog-ctrf.json)`);
+}
--- a/tests/homolog/api/core.http
+++ b/tests/homolog/api/core.http
@@ -0,0 +1,41 @@
+### GET /v1/models autenticado retorna catálogo
+GET {{baseUrl}}/v1/models
+Authorization: Bearer {{apiKey}}
+
+?? status == 200
+?? js response.parsedBody.data.length > 0
+
+### chat completions non-stream com modelo do tier crítico
+POST {{baseUrl}}/v1/chat/completions
+Authorization: Bearer {{apiKey}}
+Content-Type: application/json
+
+{
+  "model": "{{smokeModel}}",
+  "messages": [{ "role": "user", "content": "Reply with exactly: OK" }],
+  "max_tokens": 5,
+  "stream": false
+}
+
+?? status == 200
+?? js response.parsedBody.choices[0].message.content.length > 0
+
+### management sem credencial é rejeitado (401)
+# /v1/models pode ser público (REQUIRE_API_KEY off na VPS), então o teste de auth
+# usa a superfície de management, que exige credencial sempre.
+GET {{baseUrl}}/api/keys
+
+?? status == 401
+
+### management com bearer inválido é rejeitado (403)
+GET {{baseUrl}}/api/keys
+Authorization: Bearer or-invalid-key-homolog
+
+?? status == 403
+
+### health é público e saudável
+# httpYac trata o lado direito de `==` como literal — sem aspas.
+GET {{baseUrl}}/api/monitoring/health
+
+?? status == 200
+?? js response.parsedBody.status == healthy
--- a/tests/homolog/ui/api-key-flow.spec.ts
+++ b/tests/homolog/ui/api-key-flow.spec.ts
@@ -0,0 +1,31 @@
+import { test, expect } from "@playwright/test";
+
+// Locators confirmados em
+// src/app/(dashboard)/dashboard/api-manager/ApiManagerPageClient.tsx (rota real da
+// tela de keys é /dashboard/api-manager, não /dashboard/api-keys):
+//  - botão "Create API Key" (t("createKey")) abre o modal; o submit do modal tem o mesmo texto
+//  - <Input placeholder={t("keyNamePlaceholder")}> = "e.g. Production Key"
+//  - após criar, abre o "Created Key Modal" (t("keyCreated")) — fechar pelo botão t("done")="Done"
+//  - cada key vira uma linha div.grid-cols-12; o botão de deletar tem title={t("deleteKey")}="Delete key"
+//  - handleDeleteKey usa window.confirm(t("deleteConfirm")) — não é modal de UI,
+//    precisa do listener page.on("dialog", ...).
+const KEY_NAME = `homolog-ui-${Date.now()}`;
+
+test("cria e revoga uma API key pela UI", async ({ page }) => {
+  page.on("dialog", (dialog) => dialog.accept());
+
+  await page.goto("/dashboard/api-manager");
+  await page.getByRole("button", { name: "Create API Key" }).first().click();
+  await page.getByPlaceholder("e.g. Production Key").fill(KEY_NAME);
+  // segundo "Create API Key" é o submit do modal (o primeiro é o botão que o abriu)
+  await page.getByRole("button", { name: "Create API Key" }).last().click();
+
+  // fecha o modal "API Key Created"
+  await page.getByRole("button", { name: "Done" }).click();
+  const row = page.locator("div.grid-cols-12", { hasText: KEY_NAME });
+  await expect(row).toHaveCount(1);
+
+  // revoga a mesma key (cleanup — a suíte não deixa lixo na VPS)
+  await row.getByTitle("Delete key").click();
+  await expect(page.locator("div.grid-cols-12", { hasText: KEY_NAME })).toHaveCount(0);
+});
--- a/tests/homolog/ui/auth.setup.ts
+++ b/tests/homolog/ui/auth.setup.ts
@@ -0,0 +1,13 @@
+import { test as setup, expect } from "@playwright/test";
+import { STORAGE_STATE } from "./playwright.config";
+
+// Locators confirmados em src/app/login/page.tsx: <Input type="password"> dentro de um
+// <form onSubmit={handleLogin}> com <Button type="submit">{t("continue")}</Button>.
+setup("autentica e salva storageState", async ({ page }) => {
+  await page.goto("/login");
+  await page.locator('input[type="password"]').fill(process.env.HOMOLOG_ADMIN_PASSWORD!);
+  await page.locator('button[type="submit"]').click();
+  await page.waitForURL(/\/dashboard/);
+  await expect(page).toHaveURL(/dashboard/);
+  await page.context().storageState({ path: STORAGE_STATE });
+});
--- a/tests/homolog/ui/playwright.config.ts
+++ b/tests/homolog/ui/playwright.config.ts
@@ -0,0 +1,42 @@
+import { defineConfig } from "@playwright/test";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+
+const HERE = path.dirname(fileURLToPath(import.meta.url));
+export const STORAGE_STATE = path.join(HERE, ".auth", "admin.json");
+
+export default defineConfig({
+  testDir: ".",
+  timeout: 60_000,
+  retries: 1,
+  // Sem fullyParallel, os 98 testes de routes.spec.ts (mesmo arquivo) rodam
+  // SERIALIZADOS num único worker (~10min); com ele, distribuem entre os workers.
+  fullyParallel: true,
+  workers: 8,
+  reporter: [
+    ["list"],
+    [
+      // outputDir ABSOLUTO: o reporter resolve paths relativos contra o CWD do
+      // processo (não contra o config) — um path relativo escapava do worktree.
+      "playwright-ctrf-json-reporter",
+      {
+        outputDir: path.resolve(HERE, "..", "..", "..", "homolog-report"),
+        outputFile: "ui-ctrf.json",
+      },
+    ],
+  ],
+  use: {
+    baseURL: process.env.HOMOLOG_BASE_URL || "http://192.168.0.15:20128",
+    trace: "retain-on-failure",
+    screenshot: "only-on-failure",
+  },
+  projects: [
+    { name: "setup", testMatch: /auth\.setup\.ts/ },
+    {
+      name: "homolog",
+      testMatch: /.*\.spec\.ts/,
+      dependencies: ["setup"],
+      use: { storageState: STORAGE_STATE },
+    },
+  ],
+});
--- a/tests/homolog/ui/routes.spec.ts
+++ b/tests/homolog/ui/routes.spec.ts
@@ -0,0 +1,40 @@
+import { test, expect } from "@playwright/test";
+import fs from "node:fs";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+
+// Descobre as rotas estáticas do dashboard a partir do próprio repo:
+// cada page.tsx sob src/app/(dashboard)/dashboard vira uma rota; grupos (x) somem
+// do path e rotas dinâmicas [param] são puladas (sem dado real garantido).
+const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "../../..");
+const BASE = path.join(ROOT, "src", "app", "(dashboard)", "dashboard");
+
+function discoverRoutes(dir: string, prefix = "/dashboard"): string[] {
+  const routes: string[] = [];
+  if (fs.existsSync(path.join(dir, "page.tsx"))) routes.push(prefix || "/");
+  for (const e of fs.readdirSync(dir, { withFileTypes: true })) {
+    if (!e.isDirectory() || e.name.startsWith("[") || e.name.startsWith("_")) continue;
+    const seg = e.name.startsWith("(") ? "" : `/${e.name}`;
+    routes.push(...discoverRoutes(path.join(dir, e.name), `${prefix}${seg}`));
+  }
+  return [...new Set(routes)];
+}
+
+for (const route of discoverRoutes(BASE)) {
+  test(`rota ${route} carrega sem crash`, async ({ page }) => {
+    const pageErrors: string[] = [];
+    page.on("pageerror", (err) => pageErrors.push(err.message));
+
+    const res = await page.goto(route, { waitUntil: "domcontentloaded" });
+    expect(res!.status(), `HTTP em ${route}`).toBeLessThan(400);
+    // "networkidle" nunca assenta em telas com polling/websocket ao vivo (30s x 98 rotas
+    // estourava o run inteiro) — "load" + um settle curto e suficiente para hidratar e
+    // deixar um crash de client component (pageerror / error boundary) aparecer.
+    await page.waitForLoadState("load", { timeout: 10_000 }).catch(() => {});
+    await page.waitForTimeout(1_500);
+
+    // Error boundary do Next: nunca pode aparecer
+    await expect(page.locator("text=Application error")).toHaveCount(0);
+    expect(pageErrors, `pageerror em ${route}: ${pageErrors.join(" | ")}`).toHaveLength(0);
+  });
+}
--- a/tests/unit/homolog-admin-client.test.ts
+++ b/tests/unit/homolog-admin-client.test.ts
@@ -0,0 +1,17 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+import { extractJwtCookie, extractApiKey } from "../../scripts/homolog/lib/adminClient.mjs";
+
+test("extrai o cookie JWT do set-cookie do login", () => {
+  const jwt = extractJwtCookie(["auth_token=abc.def.ghi; Path=/; HttpOnly; SameSite=Lax"]);
+  assert.equal(jwt, "auth_token=abc.def.ghi");
+});
+
+test("retorna null sem set-cookie de token", () => {
+  assert.equal(extractJwtCookie(["other=1; Path=/"]), null);
+});
+
+test("extrai key e id do POST /api/keys", () => {
+  const r = extractApiKey({ key: "or-abc123", id: "k1", name: "homolog-run" });
+  assert.deepEqual(r, { key: "or-abc123", id: "k1" });
+});
--- a/tests/unit/homolog-parity.test.ts
+++ b/tests/unit/homolog-parity.test.ts
@@ -0,0 +1,29 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+import { evaluateParity } from "../../scripts/homolog/lib/parity.mjs";
+
+test("parity OK quando health bate com a versão esperada", () => {
+  const r = evaluateParity(
+    { status: "healthy", version: "3.8.49" },
+    { expectedVersion: "3.8.49", httpStatus: 200 }
+  );
+  assert.equal(r.ok, true);
+  assert.deepEqual(r.failures, []);
+});
+
+test("parity falha listando cada divergência", () => {
+  const r = evaluateParity(
+    { status: "degraded", version: "3.8.47" },
+    { expectedVersion: "3.8.49", httpStatus: 200 }
+  );
+  assert.equal(r.ok, false);
+  assert.equal(r.failures.length, 2); // status!=healthy, version mismatch
+});
+
+test("parity falha em HTTP não-200 mesmo com body bom", () => {
+  const r = evaluateParity(
+    { status: "healthy", version: "3.8.49" },
+    { expectedVersion: "3.8.49", httpStatus: 503 }
+  );
+  assert.equal(r.ok, false);
+});
--- a/tests/unit/homolog-promptfoo-ctrf.test.ts
+++ b/tests/unit/homolog-promptfoo-ctrf.test.ts
@@ -0,0 +1,18 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+import { promptfooToCtrf } from "../../scripts/homolog/lib/promptfooToCtrf.mjs";
+
+test("mapeia resultados do promptfoo para tests CTRF", () => {
+  const ctrf = promptfooToCtrf({
+    results: {
+      results: [
+        { provider: { label: "openai" }, success: true, latencyMs: 812 },
+        { provider: { label: "grok" }, success: false, latencyMs: 30000, error: "timeout" },
+      ],
+    },
+  });
+  assert.equal(ctrf.results.summary.tests, 2);
+  assert.equal(ctrf.results.summary.passed, 1);
+  assert.equal(ctrf.results.tests[1].status, "failed");
+  assert.equal(ctrf.results.tests[1].name, "provider-smoke: grok");
+});
--- a/tests/unit/homolog-provider-tiers.test.ts
+++ b/tests/unit/homolog-provider-tiers.test.ts
@@ -0,0 +1,24 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+import { pickSmokeModels } from "../../scripts/homolog/lib/providerTiers.mjs";
+
+const CATALOG = [
+  { id: "openai/gpt-5-mini" },
+  { id: "openai/gpt-5" },
+  { id: "anthropic/claude-sonnet-5" },
+  { id: "mistral/mistral-small" },
+  { id: "grok/grok-4-fast" },
+];
+
+test("1 modelo por provider crítico (o primeiro do catálogo)", () => {
+  const picks = pickSmokeModels(CATALOG, ["openai", "anthropic", "grok"]);
+  assert.deepEqual(
+    picks.map((p) => p.model),
+    ["openai/gpt-5-mini", "anthropic/claude-sonnet-5", "grok/grok-4-fast"]
+  );
+});
+
+test("provider crítico ausente do catálogo vira miss reportável", () => {
+  const picks = pickSmokeModels(CATALOG, ["openai", "nvidia"]);
+  assert.equal(picks.find((p) => p.provider === "nvidia").model, null);
+});
--- a/tests/unit/homolog-sse-parser.test.ts
+++ b/tests/unit/homolog-sse-parser.test.ts
@@ -0,0 +1,28 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+import { parseSseChunk, summarizeStream } from "../../scripts/homolog/lib/sseCheck.mjs";
+
+test("parseSseChunk separa eventos data: e detecta [DONE]", () => {
+  const events = parseSseChunk('data: {"choices":[{"delta":{"content":"O"}}]}\n\ndata: [DONE]\n\n');
+  assert.equal(events.length, 2);
+  assert.equal(events[1], "[DONE]");
+});
+
+test("parseSseChunk acha data: mesmo precedido de comment-lines SSE no mesmo bloco", () => {
+  // Formato real da VPS (v3.8.47): trailers de telemetria como comments (`: x-omniroute-*`)
+  // no MESMO bloco do data: [DONE] — o parser não pode olhar só o início do bloco.
+  const chunk =
+    'data: {"choices":[{"delta":{"content":"OK"}}]}\n\n' +
+    ": x-omniroute-cache-hit=false\n: x-omniroute-latency-ms=67\ndata: [DONE]\n\n";
+  const events = parseSseChunk(chunk);
+  assert.deepEqual(events, ['{"choices":[{"delta":{"content":"OK"}}]}', "[DONE]"]);
+});
+
+test("summarizeStream exige >=1 delta de conteúdo e terminador [DONE]", () => {
+  const good = summarizeStream(['{"choices":[{"delta":{"content":"OK"}}]}', "[DONE]"]);
+  assert.equal(good.ok, true);
+  const noDone = summarizeStream(['{"choices":[{"delta":{"content":"OK"}}]}']);
+  assert.equal(noDone.ok, false);
+  const noContent = summarizeStream(["[DONE]"]);
+  assert.equal(noContent.ok, false);
+});
				`@@ -0,0 +1 @@`
				- Homologation suite: new `npm run homolog` runs the full release-homologation battery against the deployed VPS — health/version parity, API + real SSE streaming with an ephemeral API key (created and revoked by the run), minimal-cost real-provider smoke (promptfoo generated from the live catalog), and a Playwright sweep that loads every dashboard route and exercises the API-key UI flow — emitting a unified CTRF report that backs the release STOP #2 checklist