Files
OmniRoute/tests/unit/web-session-credentials.test.ts
Diego Rodrigues de Sa e Souza 5d1ad576a1 feat(quality): gate the free-tier headline so it can never silently drift again (#7798)
* docs(free-tiers): correct the headline to the 1.37B the catalog actually computes

The 2026-06-17 honesty correction landed 1.54B, but v3.8.42 reclassified longcat
from a 150M/mo recurring grant to a one-time 10M signup credit and the doc was
never resynced. Verified against computeFreeModelTotals() at every release tag
from 3.8.13 to HEAD: no free provider was lost by mistake.

* feat(quality): gate the free-tier headline against the live catalog

The README headlined ~1.6B free tokens/mo for seven releases after the catalog had
already been corrected down to 1.37B. No gate watched that number, so the drift was
invisible — check:docs-counts only covered providers, locales, executors, strategies,
oauth, a2a skills and cloud agents.

Adds a STRICT check that runs computeFreeModelTotals() (the same function behind
/api/free-tier/summary) and fails the build when README.md or FREE_TIERS.md publish a
headline that no longer rounds to it. Degrades to a skip if tsx is unavailable rather
than going falsely red.

The extractor is a whitelist: the theoretical ceiling (~10B), the historical ~1.94B and
per-model rows (~1.00B) are legitimate figures that must never trip the gate.

Also adds the biweekly-audit note under the README headline, so readers know the number
moves both ways and is what the catalog computes rather than a rounded-up best case.

* feat(quality): extend the counts gate to engines, MCP tools/scopes and CLI tools

The v3.8.49 audit found four more numbers that had silently drifted, all invisible to
CI because check:docs-counts only watched providers/locales/executors/strategies/oauth/
a2a/cloud-agents: 10->11 compression engines, 94->104 MCP tools, 30->31 scopes,
26->33 CLI tools.

Adds a generic makeNumberClaimValidator that reads every fact in ONE tsx subprocess via
the same functions the app serves (ENGINE_IDS, countUniqueMcpTools, the live scope union,
CLI_TOOLS) — never a hardcoded copy — with DATA_DIR redirected to a throwaway dir so
importing the MCP tool modules can't touch the operator's SQLite. Each check declares a
skip pattern so legitimate non-aggregate figures never trip it: per-module tool counts
('Memory tool definitions (3 tools)') and the CLI catalog total sitting next to the MCP
total. Degrades to a skip when tsx is unavailable rather than a false red.

7 new unit tests (all pass) covering the exact stale values this audit found and proving
per-module counts are ignored.

* docs(diagrams): sync the animated cards and mermaid sources to the audited v3.8.49 numbers

The README text was fixed in #7795 but the SVG cards and mermaid sources kept the
old numbers baked in — exactly the drift the readers see first.

- compression-pipeline.svg: 10 -> 11 engine cells (Omniglyph added as #8, matching
  the README alt text), re-spaced 51px cells, highlight cascade re-timed, the
  Caveman kill-dot repositioned inside its cell, default-stack bracket recentered
- free-tier-budget.svg: bar and grid rebuilt from computeFreeModelTotals() — 21 -> 19
  countable pools (LongCat-2.0 moved to one-time credit, Inclusion provider removed),
  huggingchat entry is now ERNIE 4.5 VL, kiro shows Claude Sonnet 4.5, signup credits
  ~616M -> ~626M (+longcat 10M pill), aria said 'about 1.6 billion' -> 1.4/2.0,
  lower sections shifted up 30px (viewBox 872 -> 842)
- promise-pillars.svg: 26 -> 33 coding agents
- mcp-tools-94.mmd -> mcp-tools-104.mmd: real per-collection unique contributions
  (42 base + memory 3 + skill 4 + githubSkill 3 + pool 6 + gamification 8 + plugin 8
  + notion 6 + obsidian 22 + compression 2), exported SVG regenerated, zh-CN ref synced
- request-pipeline.mmd: 17 -> 18 strategies, exported SVG regenerated
- README free-tier alt + docs/diagrams/README.md synced to the same numbers

Both edited cards pass validate-svg.sh and were render-verified at 4 timestamps
(animation runs; first frame is the finished composition).

* docs(env): register the 4 env vars missing from the .env.example contract (base-red unblock)

FREE_PROXY_AUTO_SYNC_ENABLED / FREE_PROXY_AUTO_SYNC_INTERVAL_MS (scheduler.ts) and
MITM_ROOT_CA_ENABLED / MITM_CERT_MODE (mitm manager/server, #6684) landed on
release/v3.8.49 without their .env.example + ENVIRONMENT.md entries, turning the
docs-gates job red for every PR on the branch. Documented with their real defaults
and the set-by-manager caveat for MITM_CERT_MODE.

* fix(dashboard): narrow the Codex session ParseResult with an equality check (base-red unblock)

#7725 landed 'if (!result.ok)' in OAuthModal — under this repo's strict:false,
tsc 6 only narrows a discriminated union on the equality form, so the negation
raised TS2339 (Property 'error' does not exist on ParseResult) and turned the
dashboard-typecheck gate red for every PR on release/v3.8.49. Runtime semantics
are identical (ok is a strict boolean).

Also ratchets the frozen baseline down 260 -> 259: the real fix here plus 3
baselined errors that other merges already fixed (CostOverviewTab TS2304,
SidebarTab TS2322, FreePoolTab TS2304). Baseline diff is deletions-only.

* fix(dashboard): keep OAuthModal within the frozen file-size cap

The narrowing comment pushed the file to 1032 > 1030 frozen; the rationale lives
in the previous commit message and the dashboard-typecheck gate itself guards the
'=== false' form from being refactored back to '!result.ok'.

* test(providers): align the grok-web credential assertion with the #7567 hint (base-red unblock)

#7713 added hintKey/hintFallback (proactive cf_clearance/User-Agent guidance) to the
grok-web web-session metadata without touching this test's deepEqual, turning unit
shard 2/4 red for every PR on release/v3.8.49. Rewritten in the same contract-only
style the file already uses for lmarena: structural fields stay strictly asserted,
the hint asserts key + intent (cf_clearance / User-Agent) without freezing operator
copy. Net stronger than before — the old assertion never checked the hint at all.

* test(golden): regenerate translate-path snapshot for the notion-web endpoint move (base-red unblock)

#7768 switched notion-web to app.notion.com without regenerating the golden,
turning unit shard 3/4 red for every PR on release/v3.8.49. Two-line regen,
reflects the deliberate production change.
2026-07-19 19:07:44 -03:00

117 lines
5.3 KiB
TypeScript

import test from "node:test";
import assert from "node:assert/strict";
const providers = await import("../../src/shared/constants/providers.ts");
const webSessionCredentials =
await import("../../src/app/(dashboard)/dashboard/providers/[id]/webSessionCredentials.ts");
test("web session credential metadata covers every web-cookie provider", () => {
for (const providerId of Object.keys(providers.WEB_COOKIE_PROVIDERS)) {
assert.ok(
webSessionCredentials.getWebSessionCredentialRequirement(providerId),
`${providerId} should declare its required web-session credential`
);
}
});
test("web session credential metadata identifies cookie, token, and no-auth providers", () => {
// Grok needs BOTH sso and sso-rw cookies (#3180). #7567 added the proactive
// cf_clearance/User-Agent hint — assert its intent, don't freeze operator copy.
{
const req = webSessionCredentials.getWebSessionCredentialRequirement("grok-web");
assert.ok(req && req.kind === "cookie");
assert.equal(req.credentialName, "sso + sso-rw");
assert.equal(req.placeholder, "sso=...; sso-rw=...");
assert.equal(req.acceptsFullCookieHeader, true);
assert.deepEqual(req.storageKeys, ["cookie", "sso", "sso-rw"]);
assert.equal(req.hintKey, "grokWebCookieHint");
assert.ok(typeof req.hintFallback === "string" && /cf_clearance/.test(req.hintFallback));
assert.ok(/User-Agent/.test(req.hintFallback));
}
assert.deepEqual(webSessionCredentials.getWebSessionCredentialRequirement("copilot-web"), {
kind: "token",
credentialName: "access_token",
placeholder: "access_token=... or a DevTools HAR export",
acceptsFullCookieHeader: false,
storageKeys: ["token", "access_token", "accessToken"],
});
assert.deepEqual(webSessionCredentials.getWebSessionCredentialRequirement("deepseek-web"), {
kind: "token",
credentialName: "userToken",
placeholder: "userToken=... or paste raw userToken",
acceptsFullCookieHeader: false,
storageKeys: ["token", "userToken"],
});
// Arena (lmarena): assert contract/intent only — do not freeze UX copy.
// #3810 chunk name, #4271 split SSR cookies, full Cookie header paste.
{
const req = webSessionCredentials.getWebSessionCredentialRequirement("lmarena");
assert.ok(req && req.kind === "cookie");
assert.equal(req.acceptsFullCookieHeader, true);
assert.equal(req.hintKey, "lmarenaWebCookieHint");
assert.ok(req.storageKeys.includes("cookie"));
assert.ok(req.storageKeys.includes("arena-auth-prod-v1.0"));
assert.ok(req.storageKeys.includes("arena-auth-prod-v1.1"));
// legacy key retained for already-saved credentials
assert.ok(req.storageKeys.includes("session"));
assert.ok(/full cookie header/i.test(req.credentialName));
assert.ok(/arena-auth-prod-v1/i.test(req.placeholder));
// hintFallback is operator copy — may change with CF/reCAPTCHA notes; must still
// steer users to a full header and away from the empty single base cookie.
assert.ok(typeof req.hintFallback === "string" && req.hintFallback.length > 0);
assert.ok(/full cookie header/i.test(req.hintFallback));
assert.ok(/arena-auth-prod-v1/i.test(req.hintFallback));
assert.ok(/empty/i.test(req.hintFallback));
}
assert.deepEqual(webSessionCredentials.getWebSessionCredentialRequirement("huggingchat"), {
kind: "cookie",
credentialName: "full Cookie header (hf-chat + token)",
placeholder:
"hf-chat=...; token=...; aws-waf-token=... (full Cookie header from huggingface.co)",
acceptsFullCookieHeader: true,
storageKeys: ["cookie", "hf-chat"],
});
// veoaifree-web is now a NOAUTH provider — not in WEB_SESSION_CREDENTIAL_REQUIREMENTS
assert.equal(webSessionCredentials.getWebSessionCredentialRequirement("veoaifree-web"), null);
assert.deepEqual(webSessionCredentials.getWebSessionCredentialRequirement("t3-web"), {
kind: "cookie",
credentialName: "convex-session-id + Cookie header",
placeholder: "convex-session-id=abc123...; Cookie: ...",
acceptsFullCookieHeader: true,
storageKeys: ["cookie", "convex-session-id", "convexSessionId"],
// #5465 — t3.chat ships a step-by-step DevTools copy hint (localStorage + Cookie header).
hintKey: "t3ChatWebCookieHint",
});
});
test("web session credential validator requires provider-specific non-empty values", () => {
assert.equal(
webSessionCredentials.hasUsableWebSessionCredential("qwen-web", { token: "qwen-token" }),
true
);
assert.equal(
webSessionCredentials.hasUsableWebSessionCredential("qwen-web", { token: " " }),
false
);
assert.equal(
webSessionCredentials.hasUsableWebSessionCredential("qwen-web", { unrelated: "value" }),
false
);
assert.equal(
webSessionCredentials.hasUsableWebSessionCredential("chatgpt-web", {
cookie: "__Secure-next-auth.session-token=session",
}),
true
);
assert.equal(
webSessionCredentials.hasUsableWebSessionCredential("chatgpt-web", { unrelated: "value" }),
false
);
});
test("no-auth web providers can be saved without an API key", () => {
assert.equal(providers.providerAllowsOptionalApiKey("veoaifree-web"), true);
assert.equal(webSessionCredentials.requiresWebSessionCredential("veoaifree-web"), false);
assert.equal(webSessionCredentials.requiresWebSessionCredential("chatgpt-web"), true);
});