Files
OmniRoute/tests/unit/provider-validation-specialty.test.ts
Diego Rodrigues de Sa e Souza fa367dd99e Release v3.8.27 (#3968)
* chore(release): open v3.8.27 development cycle

* fix(security): polynomial ReDoS in comboAgentMiddleware regex (#3982)

* fix(security): eliminate polynomial ReDoS in comboAgentMiddleware <omniModel> regex (CodeQL js/polynomial-redos)

CACHE_TAG_PATTERN wrapped the tag in an unbounded `(?:\\n|\n|\r)*` prefix/suffix.
On an unanchored `.test()`/`.exec()` that is O(n²) on inputs with many newlines
(CodeQL js/polynomial-redos, alerts #612/#613). The surrounding runs are irrelevant
to detecting/capturing the tag, so the detection pattern now matches only the core
`<omniModel>([^<]+)</omniModel>`; the global strip pattern still consumes the
wrapping newlines (combo.ts streaming, #531) but BOUNDED ({0,16}) so it stays linear.

Behavior preserved: detection, model extraction, multi-tag stripping (#454) and
blank-line cleanup all unchanged (107 related tests green). Adds ReDoS-safety
regression tests (50k-newline inputs complete in <1ms).

* docs(changelog): add #3982 ReDoS fix to [3.8.27]

* ci(security): harden workflows — artipacked persist-credentials + cache-poisoning + SC2086 (#3965)

* Refine provider quota card display (#3969)

Integrated into release/v3.8.27

* feat: add sidebar group separator toggles (#3971)

Integrated into release/v3.8.27

* Gate control-plane proxy direct fallback (#3963)

Integrated into release/v3.8.27

* Capture actual upstream provider requests (#3941)

Integrated into release/v3.8.27

* ci(quality): flip require-tighten + osv + Trivy to blocking (v3.8.27 cycle-end) (#3984)

* fix(resilience): respect connection cooldown stored as numeric epoch (#3954) (#3995)

rate_limited_until is a TEXT column, but setConnectionRateLimitUntil (Antigravity full-quota path) persists a raw epoch number that SQLite coerces to a numeric string ("1781696905131.0"). The selection predicate isAccountUnavailable then did new Date("1781696905131.0") -> NaN, so the cooling connection was never skipped and the router kept dispatching to rate-limited accounts. Normalize numeric-epoch strings (and number/Date/ISO) via a shared cooldownUntilMs() helper in isAccountUnavailable / getEarliestRateLimitedUntil / filterAvailableAccounts / parseFutureDateMs. ISO behavior preserved.

* fix(providers): fetch live /models for LLM7 and BytePlus (#3976) (#3996)

llm7 and byteplus carry a real modelsUrl but were not classified by any live-fetch branch of the model-import route, so their hardcoded 4-entry registry catalog was served (source local_catalog) instead of the upstream catalog. Add both to NAMED_OPENAI_STYLE_PROVIDERS so the route probes <baseUrl>/models and serves the live list, falling back to the local catalog only on fetch failure.

* fix(dashboard): logs auto-refresh reads live visibility, not a stale mount ref (#3972) (#3997)

The auto-refresh interval gated each tick on visibleRef, seeded once at mount and updated only by a visibilitychange event. A tab mounted while document.visibilityState is 'hidden' (background load, bfcache, embedded/proxied webviews) with no later visibilitychange left the ref false forever, so the interval ticked but never fetched — only the manual button worked. Read the live document.visibilityState in the tick instead.

* feat(compression): add Indonesian caveman rules and language pack (#3975)

Integrated into release/v3.8.27

(cherry picked from commit c9b5b1a892)

* fix(combo): shuffle strict-random fallback remainder to spread load (#3959) (#3998)

strict-random shuffled only the deck-selected slot 0 and left the fallback remainder in fixed priority order, so after a failing deck pick the chain always fell through to the same top-priority model — a persistently-failing model was retried on essentially every request and fallback load never spread across peers. Shuffle the remainder too (like the random strategy).

* Add provider auth visibility controls (#3953)

Integrated into release/v3.8.27

* fix(claude): forward client tool-search-tool anthropic-beta on the Claude OAuth path (#3974) (#3999)

The client-negotiated anthropic-beta: tool-search-tool-2025-10-19 was dropped on both Claude code paths (default executor rebuilt from static ANTHROPIC_BETA_CLAUDE_OAUTH; selectBetaFlags only read the client beta to gate thinking/effort), so claude.ai rejected deferred-tool requests with 400 'Tool reference not found'. Add an allowlist-merge (mergeClientAnthropicBeta) that unions the client's allowlisted betas into the outbound set on both paths, preserving #3415 (no forced thinking/effort).

* feat(providers): add model search filter to provider dashboard (#3950)

Integrated into release/v3.8.27

* fix(vision-bridge): force bridge for tokenrouter deepseek models (#3946)

Integrated into release/v3.8.27

* fix(executor): strip stream_options on non-streaming requests (#3884) (#4000)

Clients that send stream_options:{include_usage:true} regardless of stream (e.g. the OpenAI Python SDK) had it passed through on non-streaming calls; NVIDIA NIM rejected it with 400 'Stream options can only be defined when stream=True'. DefaultExecutor.transformRequest only injected/cleared stream_options on the streaming branch and never stripped a client-sent value when stream=false. Add a !stream strip branch; the streaming injection path is unchanged. Global to openai-compat providers.

* fix(qwen-web): cookie validation false-positive - check response body for user object (#3958)

Integrated into release/v3.8.27

* fix(db): persist backup retention days (#3970)

Integrated into release/v3.8.27

* 大量UI显示和i18n优化 (#3973)

Integrated into release/v3.8.27

* deps: bump the npm_and_yarn group across 1 directory with 2 updates (#3943)

Integrated into release/v3.8.27

* deps: bump form-data from 4.0.5 to 4.0.6 (#3944)

Integrated into release/v3.8.27

* deps: bump vite from 8.0.5 to 8.0.16 (#3942)

Integrated into release/v3.8.27

* chore(quality): re-baseline validation.ts 4407->4428 (#3958 qwen body-check)

The qwen-web validation body-check merged in #3958 pushed validation.ts past its
frozen size on the integrated release tip. Bump the baseline with justification;
no logic is separately extractable from the existing qwen-web validation branch.

* deps: bump the production group with 13 updates (#3915)

Integrated into release/v3.8.27 — low-risk group (playwright 1.60→1.61 minor + transitive patches; fumadocs-core 16.9→16.10 minor).

* chore(deps): ignore jscpd major bumps (v5 Rust rewrite breaks the duplication gate)

Our duplication ratchet (scripts/check/check-duplication.mjs) is pinned to jscpd@4
and parses jscpd-report.json against a frozen baseline. jscpd v5 is a native Rust
binary with no Node.js API and a different report/bin, so a major bump would break
the gate. Migrate deliberately, not via dependabot. Closes the noise from #3916.

* fix(perplexity-web): parse schematized diff_block stream so answers aren't empty (#4001)

Integrated into release/v3.8.27 — schematized diff_block parsing follow-up to #3938.

* refactor: modularize providerRegistry.ts into 159 individual provider plugins (#3993)

Modularize provider registry (#3594). Integrated into release/v3.8.27 after rebase + behavior-preservation verification (provider-consistency gate 159/232/0, typecheck, registry tests, build 556/556).

Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>

* fix(registry): restore byteplus + mimocode dropped by #3993 modularization

The provider-registry modularization (#3993) was cut from a base predating the
byteplus (#3877) and mimocode (#3837) registry entries, so merging it silently
dropped both providers (getRegistryEntry returned undefined → validation reported
'not supported'). Re-add them as registry modules in the new structure; registered
count 159→161, provider-consistency 161/232/0.

Also align the pre-existing qwen-web validator test to #3958: since the validator
now requires a real `user` object in the 200 body, the mock must carry one.

* refactor: modularize schemas (non-stacked) (#3988)

Modularize validation schemas (#3594). Integrated into release/v3.8.27 after rebase (reconciled the merged hiddenSidebarGroupLabels #3971 + intelligenceSyncRequestSchema into the new modules) + behavior verification (typecheck, 195 schema/settings/validation tests, build 556/556).

Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>

* fix(default-executor): honor custom providerSpecificData.baseUrl for OpenAI-format providers (#4002)

Integrated into release/v3.8.27 — honor custom providerSpecificData.baseUrl in DefaultExecutor (openai-format), tested.

* feat(openai): honor custom base URL in model discovery + complete openai/codex pricing (#4005)

Integrated into release/v3.8.27 — openai model-discovery honors custom base URL (SSRF-guarded) + pricing rows for new openai/codex models. Tested + baselines bumped.

* fix(live-ws): bridge sidecar events to dashboard (#4004)

Integrated into release/v3.8.27 — repair LiveWS sidecar (startup, same-origin /live-ws, main→sidecar compression.completed bridge, early-msg queue). Fixed the cookie-parse regex (\s) + added a focused unit test; baseline bumped for the non-blocking chatCore bridge.

* docs(troubleshooting): note MITM proxy cannot intercept Windows-host apps under WSL (#4003)

Integrated into release/v3.8.27 — MITM/WSL troubleshooting note.

* fix(repo): untrack accidentally-committed root node_modules symlink + gitignore it

A worktree node_modules symlink (-> the main checkout's node_modules) was staged by a
`git add -A` during the #3988 merge and committed into 05213ac6a. The symlink points
at the repo's own node_modules path, so checking it out turns the main checkout's
node_modules into a self-referential symlink (breaking tsx/all node ops). Untrack it and
add a root-anchored /node_modules ignore so the symlink form can't be re-committed (the
existing 'node_modules/' only matches directories).

* fix(quality): allowlist socks dep (declared by #4004, never allowlisted)

socks@^2.8.7 was added to package.json in #4004 (LiveWS sidecar, 02302131f)
as a phantom-dep cleanup but never added to dependency-allowlist.json, so
check:deps has been red on the release tip ever since. socks is the standard
SOCKS proxy client (dep of fetch-socks), legitimate and years old.

* feat(sse): real LLMLingua-2 ONNX compression engine (stable) (#4014)

Integrated into release/v3.8.27.

Adjustments before merge:
- Synced with the current release tip (was 11 commits behind).
- Added the 3 LLMLingua-2 ONNX optional-runtime deps to dependency-allowlist.json
  (@atjsh/llmlingua-2, @tensorflow/tfjs, js-tiktoken) — the only gate that was red.
- socks was allowlisted directly on release (separate fix d7db5c73d; it was declared
  by #4004 but never allowlisted, leaving check:deps red release-wide).

Verified locally: check:deps OK, file-size OK, public-creds OK, provider-consistency
161/232/0, typecheck:core clean, 24/24 LLMLingua tests pass. The only remaining Fast-QG
red is the pre-existing #3972 orphan test (request-logger-autorefresh-visibility-3972.test.tsx),
which is release-wide and unrelated to this PR.

* test(dashboard): rehome #3972 logs auto-refresh test so a runner collects it

tests/unit/request-logger-autorefresh-visibility-3972.test.tsx (added by #3972
via #3997) sat at the top level of tests/unit/ as a .tsx vitest test, which NO
runner collects: the node runner only globs *.test.ts, and test:vitest:ui only
runs tests/unit/ui. So the #3972 regression guard never executed in CI and
check:test-discovery was red release-wide. Move it under tests/unit/ui/ (the
collected vitest:ui path) and fix the relative import depth. Verified: the test
now runs and passes (2/2), and check:test-discovery is green.

* feat(compression): capture per-engine analytics (#3960) + Lite schema fix (#3952) (#4018)

Captures the net-new value from #3960 (per-engine breakdown analytics) and #3952 (Lite engine schema fix) onto release/v3.8.27. Fast QG green; 622/622 compression+analytics tests pass.

* fix(sse): guard model-less registry entries in getUnsupportedParams (mimocode) (#4015)

Real bugfix: guard model-less registry entries (mimocode) in getUnsupportedParams so handleChatCore no longer throws 'entry.models is not iterable' / reports 'All models failed' for unrelated requests. Includes a regression test. Fast QG green.

* feat(ci): Quality Gate v2 — Onda 0 + Onda 1 (gate flips, TIA, SAST, DAST-smoke, mutation infra) (#4016)

* docs(ops): add quality-gate assessment + replication playbook (Fase 9 foundation)

* feat(ci): flip oasdiff breaking-change gate to blocking (ratchet)

* docs(ops): deliver main branch-protection ruleset for owner to apply

* fix(ci): run typecheck:core in PR->release fast-gates (close fast-gates hole, part 1)

* perf(mutation): enable Stryker incremental mode + cache (scales the 60/80 rollout)

* feat(ci): commit CodeQL advanced config (security-extended), replacing default-setup

* feat(ci): version semgrep SAST workflow (owasp/secrets), advisory

* feat(quality): TIA test-impact map builder (import-graph; map built at runtime, gitignored)

* feat(quality): TIA impacted-test selector with run-all fail-safe

* fix(ci): run TIA-impacted unit tests in PR->release fast-gates (build map at runtime, fail-safe full)

* feat(ci): DAST-smoke per-PR (schemathesis subset + promptfoo injection-guard, blocking)

* fix(ci): unbreak Fase 9 PR CI (MDX frontmatter, CodeQL conflict, dast-smoke advisory)

- Add MDX frontmatter to docs/ops/{BRANCH_PROTECTION_MAIN,QUALITY_GATE_PLAYBOOK}.md.
  fumadocs rejects frontmatter-less docs -> 'npm run build' failed -> broke dast-smoke's
  build step (the release fast-gates never runs build, so this only surfaced on the PR).
- codeql.yml: workflow_dispatch-only until the owner switches repo CodeQL Default->Advanced
  (advanced configs cannot be processed while default setup is enabled; documented inline).
- dast-smoke.yml: job-level continue-on-error (advisory) so this brand-new gate matures
  before it blocks (repo convention: advisory -> blocking).

* ci(quality): make TIA unit-test step advisory until release test-debt is cleared

release/v3.8.27 carries ~17 pre-existing failing unit tests (budget #3537, apiKey
#3552, several Zod schemas, Puter/Qwen executors, mimocode entry, etc.) unrelated to
this PR — the new 'run tests on PR->release' gate surfaced them. Per the repo's
advisory->blocking convention, this step enters advisory (it still runs + reports)
so pre-existing debt doesn't block the gate program. typecheck:core stays blocking.
Flip to blocking (remove continue-on-error) once the release suite is green.

* fix(sse): preserve Kiro streaming finish_reason tool_calls (#3980) (#4025)

* fix(guardrails): preserve original image when vision-bridge describe fails (#4012) (#4026)

* feat(api): advertise combo capabilities on import surfaces (#3979) (#4027)

* feat(sse): delegated Anthropic Context Editing for Claude (clear_tool_uses) (#4021)

Opt-in Claude-only delegated compression: injects context_management.clear_tool_uses_20250919 at the Claude pre-serialization chokepoint (composes with clear_thinking, thinking first), threaded via ExecuteInput from handleChatCore. Pure edit-builder + 11 tests (7 unit + 4 e2e fetch-capture). Beta context-management-2025-06-27 already advertised; allowlist done. Telemetry/400-fallback/claude-web coverage deferred.

* fix(opencode): map x-session-affinity to x-opencode-session for custom providers (#4022) (#4028)

* fix(dashboard): Playground Compare tab loading + HTTP method guard (#4024)

randomUUID non-HTTPS fallback + static CompareTab import; raw HTTP TRACE->405 method guard wired into dev + standalone servers. Integrated into release/v3.8.27.

* refactor(dashboard): settings UI layout + API Keys naming (#4020)

Presentation/relabel refactor of the Settings dashboard (API Manager -> API Keys), card relocations, Toggle adoption, present-but-disabled engine steps. Auth-file changes are string/comment-only (no behavior change). Integrated into release/v3.8.27.

* fix: restore unit regressions dropped by lossy schema/registry modularizations (#4030)

Restores schema fields (combo reasoningTokenBuffer, budget-0 #3537, openrouter preset, proxy family #3777, resilience degradation/providerCooldown), qwen-web v2 endpoint+catalog, mimocode models key — all dropped by #3988/#3993 — and aligns 3 tests to #3941/#3993. Verified: 8 failing regression tests on release tip -> 131/131 green on this branch. Integrated into release/v3.8.27.

* fix(api): return 400 (not 500) for malformed JSON on /api/auth/login (#4031)

Wrap request.json() so a malformed/non-JSON login body returns a structured 400 instead of falling through to the 500 catch. Fixes the schemathesis high-risk-endpoint DAST finding (verified: schemathesis step now passes). +TDD test. Integrated into release/v3.8.27.

* feat(dashboard): real circuit-breaker state in the Combo Live cascade (U1b) (#4029)

Overlays real provider circuit-breaker state (GET /api/monitoring/health) onto the Combo Live cascade as a 'CB: OPEN · 41s' badge. Pure enrichRunWithBreakers + fail-soft useProviderBreakerHealth poll; graceful when health is absent. +13 tests. Integrated into release/v3.8.27.

* Fix promptfoo security assertion parsing (#4032)

* chore(deps): dependabot security bumps + drop unused gray-matter (#4036)

Integrated into release/v3.8.27 — dependabot security bumps (form-data/js-yaml/protobufjs/dompurify/hono) + drop unused gray-matter. Unblocks the npm audit:deps gate (Lint) branch-wide.

* fix(ci): scope TIA to node:test unit files only (mirror test:unit glob) (#4035)

Integrated into release/v3.8.27 — scopes the advisory TIA step to the test:unit node:test glob, fixing the 99 false failures. +4 TDD.

* Refine compression settings, storage labels, and sidebar grouping (#4033)

Integrated into release/v3.8.27 — relocate Token Saver into Compression Settings (controlled component), reorder Security/Authz tabs, storage labels + i18n relabel. Thanks @rdself!

* [codex] add per-key local usage command (#4034)

Integrated into release/v3.8.27 — per-key local @@om-usage command (cached quota, no upstream routing). Rebased onto modularized schemas/keys.ts + file-size rebaseline. Thanks @Witroch4!

* chore(release): reconcile v3.8.27 CHANGELOG + i18n mirrors

* ci(quality): unblock v3.8.27 release gates (zizmor pin + test-masking allowlist)

- zizmor ratchet (151→139, no regression): SHA-pin every action ref ADDED this
  cycle — codeql/dast-smoke/semgrep (3 new workflows) + trivy-action (docker-publish)
  + actions/cache (nightly-mutation). Pre-existing tag refs keep the repo convention.
- test-masking: add config/quality/test-masking-allowlist.json + allowlist support in
  check-test-masking.mjs (exempts ONLY the net-assert-reduction signal; tautology/skip/
  deletion still fire). Allowlists 2 verified-legitimate reductions:
  appearance-widget-settings-schema (#4033 removed showTokenSaverOnEndpoint field) and
  dashboard-shell-tabs (#3973 tabs→redirect refactor, asserts replaced). +4 gate tests.

* test(quality): reword test-masking self-test comments to avoid literal masking patterns

The added allowlist-test comments contained the literal strings 'assert.ok(true)' and
'.skip' which the masking detector's own regexes match as text — making the gate flag
its own test file (net +1 tautology/skip/extended-tautology vs main). Reworded to plain
prose ('a new tautology', 'a new skip marker'); test logic unchanged (24/24 pass).

* fix(quality): unblock v3.8.27 release — align 3 stale tests + restore modularized settings-schema parity

Release-PR full CI surfaced 3 deterministic test failures (no live product regression),
all stale vs legitimate cycle changes:

- settings-schema parity (#3988): the modularized updateSettingsSchema barrel
  (schemas/settings.ts) had diverged from the canonical settingsSchemas.ts (45 vs 85
  fields — 40 dropped + 6 extra), a lossy-modularization dead-code copy. Re-export from
  the canonical source so the barrel can never diverge again (runtime already uses
  canonical). Parity test now passes.
- api-manager permissions modal: #4034 added a 4th self-service switch (per-key usage
  allowance); a11y invariant (every switch type="button") still holds. Updated the
  static count 3 -> 4.
- pack-artifact policy: dist/http-method-guard.cjs became a required runtime path;
  added it to the test's expected missing-paths list.

Also documents the gate gap for Fase 9 (QUALITY_GATE_PLAYBOOK Parte 6): G1 run the
deterministic unit layer + test-masking on PR->release (not just PR->main), G2 a
modularization-parity gate (would have caught the #3988 drop at its PR), G3 flake
quarantine. Env flakes (LiveWS startup timeout, integration server-startup cascade)
are pre-existing/CI-env, triaged separately.

---------

Co-authored-by: Randi <55005611+rdself@users.noreply.github.com>
Co-authored-by: Veier04 <118300867+Veier04@users.noreply.github.com>
Co-authored-by: Felipe Sartori <felipesartori.ti@gmail.com>
Co-authored-by: WormAlien <164898390+WormAlien@users.noreply.github.com>
Co-authored-by: thezukiru <121331256+thezukiru@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: NOXX - Commiter <artur1992123@mail.ru>
Co-authored-by: Paijo <14921983+oyi77@users.noreply.github.com>
Co-authored-by: Demiurge The Single <megamen932@gmail.com>
Co-authored-by: Witroch4 <witalo_rocha@hotmail.com>
2026-06-17 02:43:21 -03:00

2752 lines
100 KiB
TypeScript

import test from "node:test";
import assert from "node:assert/strict";
const {
validateProviderApiKey,
validateClaudeCodeCompatibleProvider,
validateCommandCodeProvider,
isSecurityBlockError,
} = await import("../../src/lib/providers/validation.ts");
const { SafeOutboundFetchError } = await import("../../src/shared/network/safeOutboundFetch.ts");
const { __setTlsFetchOverrideForTesting: __setPplxTlsFetchOverride } =
await import("../../open-sse/services/perplexityTlsClient.ts");
const { __setTlsFetchOverrideForTesting: __setGrokTlsFetchOverride } =
await import("../../open-sse/services/grokTlsClient.ts");
const { COMMAND_CODE_VERSION } = await import("../../open-sse/executors/commandCode.ts");
const originalFetch = globalThis.fetch;
test.afterEach(() => {
globalThis.fetch = originalFetch;
__setPplxTlsFetchOverride(null);
__setGrokTlsFetchOverride(null);
});
function toPlainHeaders(headers: any) {
if (headers instanceof Headers) return Object.fromEntries(headers.entries());
return Object.fromEntries(
Object.entries(headers || {}).map(([key, value]) => [key, String(value)])
);
}
function metaAiSseText(content: string, streamingState = "DONE") {
return `event: next
data: ${JSON.stringify({
data: {
sendMessageStream: {
__typename: "AssistantMessage",
id: "meta-msg-1",
content,
streamingState,
error:
streamingState === "ERROR"
? { message: content, code: null, stack: "Error: " + content }
: null,
contentRenderer: { __typename: "TextContentRenderer", text: content },
},
},
})}
event: complete
data:
`;
}
test("specialty provider validators cover Deepgram, AssemblyAI, ElevenLabs and Inworld branches", async () => {
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
const headers = init.headers || {};
if (target.match(/deepgram/i)) {
assert.equal(headers.Authorization, "Token dg-key");
return new Response(JSON.stringify({ ok: true }), { status: 200 });
}
if (target.match(/assemblyai/i)) {
assert.equal(headers.Authorization, "aa-key");
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 403 });
}
if (target.match(/elevenlabs/i)) {
return new Response(JSON.stringify({ voices: [] }), { status: 200 });
}
if (target.match(/inworld/i)) {
return new Response(JSON.stringify({ error: "bad request" }), { status: 400 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const deepgram = await validateProviderApiKey({ provider: "deepgram", apiKey: "dg-key" });
const assembly = await validateProviderApiKey({ provider: "assemblyai", apiKey: "aa-key" });
const eleven = await validateProviderApiKey({ provider: "elevenlabs", apiKey: "el-key" });
const inworld = await validateProviderApiKey({ provider: "inworld", apiKey: "iw-key" });
assert.equal(deepgram.valid, true);
assert.equal(assembly.error, "Invalid API key");
assert.equal(eleven.valid, true);
assert.equal(inworld.valid, true);
});
test("validateCommandCodeProvider ignores caller baseUrl and chatPath overrides", async () => {
globalThis.fetch = async (url, init = {}) => {
assert.equal(String(url), "https://api.commandcode.ai/alpha/generate");
const headers = init.headers as Record<string, string>;
assert.equal(headers.Authorization, "Bearer cc-key");
const body = JSON.parse(String(init.body));
assert.equal(body.params.model, "command-code-validation-model");
return new Response(JSON.stringify({ ok: true }), { status: 200 });
};
const result = await validateCommandCodeProvider({
apiKey: "cc-key",
providerSpecificData: {
baseUrl: "https://evil.example/api",
chatPath: "/v1/chat/completions",
validationModelId: "command-code-validation-model",
},
});
assert.equal(result.valid, true);
});
test("validateCommandCodeProvider defaults probe model to DeepSeek flash", async () => {
globalThis.fetch = async (_url, init = {}) => {
const body = JSON.parse(String(init.body));
assert.equal(body.params.model, "deepseek/deepseek-v4-flash");
return new Response("", { status: 400 });
};
const result = await validateCommandCodeProvider({ apiKey: "cc-key" });
assert.deepEqual(result, { valid: true, error: null });
});
test("specialty providers surface network failures and non-auth upstream failures", async () => {
globalThis.fetch = async (url) => {
const target = String(url);
if (target.match(/deepgram/i)) {
throw new Error("deepgram offline");
}
if (target.match(/elevenlabs/i)) {
return new Response(JSON.stringify({ error: "server" }), { status: 500 });
}
if (target.match(/inworld/i)) {
return new Response(JSON.stringify({ error: "forbidden" }), { status: 403 });
}
if (target.match(/longcat/i)) {
throw new Error("longcat offline");
}
throw new Error(`unexpected fetch: ${target}`);
};
const deepgram = await validateProviderApiKey({ provider: "deepgram", apiKey: "dg-key" });
const eleven = await validateProviderApiKey({ provider: "elevenlabs", apiKey: "el-key" });
const inworld = await validateProviderApiKey({ provider: "inworld", apiKey: "iw-key" });
const longcat = await validateProviderApiKey({ provider: "longcat", apiKey: "lc-key" });
assert.equal(deepgram.error, "deepgram offline");
assert.equal(eleven.error, "Validation failed: 500");
assert.equal(inworld.error, "Invalid API key");
assert.equal(longcat.error, "longcat offline");
});
test("embedding and rerank specialty validators cover Voyage AI and Jina AI", async () => {
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
if (target === "https://api.voyageai.com/v1/embeddings") {
assert.equal((init.headers as Record<string, string>).Authorization, "Bearer voyage-key");
const body = JSON.parse(String(init.body));
assert.equal(body.model, "voyage-4-large");
return new Response(JSON.stringify({ data: [{ embedding: [0.1, 0.2] }] }), { status: 200 });
}
if (target === "https://api.jina.ai/v1/rerank") {
assert.equal((init.headers as Record<string, string>).Authorization, "Bearer jina-key");
const body = JSON.parse(String(init.body));
assert.equal(body.model, "jina-reranker-v3");
return new Response(JSON.stringify({ results: [{ index: 0, relevance_score: 0.99 }] }), {
status: 200,
});
}
throw new Error(`unexpected fetch: ${target}`);
};
const voyage = await validateProviderApiKey({ provider: "voyage-ai", apiKey: "voyage-key" });
const jina = await validateProviderApiKey({ provider: "jina-ai", apiKey: "jina-key" });
assert.equal(voyage.valid, true);
assert.equal(jina.valid, true);
});
test("AWS Polly specialty validator signs DescribeVoices with SigV4", async () => {
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
const headers = init.headers as Record<string, string>;
assert.equal(target, "https://polly.us-east-2.amazonaws.com/v1/voices?Engine=standard");
assert.match(
headers.Authorization,
/^AWS4-HMAC-SHA256 Credential=AKIA_POLLY\/\d{8}\/us-east-2\/polly\/aws4_request,/
);
assert.equal(headers.host, "polly.us-east-2.amazonaws.com");
assert.equal(headers["x-amz-content-sha256"].length, 64);
return new Response(JSON.stringify({ Voices: [] }), { status: 200 });
};
const result = await validateProviderApiKey({
provider: "aws-polly",
apiKey: "aws-secret",
providerSpecificData: {
accessKeyId: "AKIA_POLLY",
region: "us-east-2",
},
});
assert.equal(result.valid, true);
});
test("AWS Polly specialty validator requires an access key id", async () => {
const result = await validateProviderApiKey({
provider: "aws-polly",
apiKey: "aws-secret",
providerSpecificData: {
region: "us-east-2",
},
});
assert.equal(result.error, "Missing AWS accessKeyId");
});
test("embedding and rerank specialty validators surface auth failures for Voyage AI and Jina AI", async () => {
globalThis.fetch = async (url) => {
const target = String(url);
if (target === "https://api.voyageai.com/v1/embeddings") {
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
}
if (target === "https://api.jina.ai/v1/rerank") {
return new Response(JSON.stringify({ error: "forbidden" }), { status: 403 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const voyage = await validateProviderApiKey({ provider: "voyage-ai", apiKey: "voyage-key" });
const jina = await validateProviderApiKey({ provider: "jina-ai", apiKey: "jina-key" });
assert.equal(voyage.error, "Invalid API key");
assert.equal(jina.error, "Invalid API key");
});
test("gitlab specialty validator accepts PAT auth on the direct access endpoint", async () => {
globalThis.fetch = async (url, init = {}) => {
assert.equal(String(url), "https://gitlab.com/api/v4/code_suggestions/direct_access");
assert.equal((init.headers as Record<string, string>).Authorization, "Bearer glpat-test");
return new Response(JSON.stringify({ token: "short-lived" }), { status: 200 });
};
const result = await validateProviderApiKey({ provider: "gitlab", apiKey: "glpat-test" });
assert.equal(result.valid, true);
});
test("gitlab specialty validator treats 401 as invalid PAT", async () => {
globalThis.fetch = async () =>
new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
const result = await validateProviderApiKey({ provider: "gitlab", apiKey: "glpat-bad" });
assert.equal(result.error, "Invalid API key");
});
test("web-cookie provider validators accept valid Grok, Perplexity, Blackbox and Muse Spark session cookies", async () => {
const calls = [];
// Grok now uses tlsFetchGrok (TLS-impersonating client) to bypass Cloudflare Enterprise.
let grokTlsCall: { url: string; options: Record<string, unknown> } | null = null;
__setGrokTlsFetchOverride(async (url, options) => {
grokTlsCall = { url, options };
return { status: 200, headers: new Headers(), text: null, body: null };
});
// Perplexity now uses tlsFetchPerplexity (TLS-impersonating client) instead of globalThis.fetch
// to bypass Cloudflare Enterprise. Use the test-only override hook to intercept calls.
let pplxTlsCall: { url: string; options: Record<string, unknown> } | null = null;
__setPplxTlsFetchOverride(async (url, options) => {
pplxTlsCall = { url, options };
return { status: 200, headers: new Headers(), text: null, body: null };
});
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
calls.push({ url: target, init });
if (target.includes("app.blackbox.ai/api/auth/session")) {
return new Response(
JSON.stringify({
user: { id: "bb-user-1", email: "premium@example.com" },
}),
{ status: 200 }
);
}
if (target.includes("app.blackbox.ai/api/check-subscription")) {
return new Response(
JSON.stringify({
hasActiveSubscription: true,
isTrialSubscription: false,
plan: "pro",
}),
{ status: 200 }
);
}
if (target.includes("meta.ai/api/graphql")) {
return new Response(metaAiSseText("Muse Spark says hello"), {
status: 200,
headers: { "Content-Type": "text/event-stream" },
});
}
throw new Error(`unexpected fetch: ${target}`);
};
const grok = await validateProviderApiKey({ provider: "grok-web", apiKey: "sso=grok-cookie" });
const perplexity = await validateProviderApiKey({
provider: "perplexity-web",
apiKey: "__Secure-next-auth.session-token=pplx-cookie",
});
const blackbox = await validateProviderApiKey({
provider: "blackbox-web",
apiKey: "__Secure-authjs.session-token=bb-cookie",
});
const museSpark = await validateProviderApiKey({
provider: "muse-spark-web",
apiKey: "abra_sess=meta-cookie",
});
assert.equal(grok.valid, true);
assert.equal(perplexity.valid, true);
assert.equal(blackbox.valid, true);
assert.equal(museSpark.valid, true);
const blackboxSessionCall = calls.find((call) =>
call.url.includes("app.blackbox.ai/api/auth/session")
);
const blackboxSubscriptionCall = calls.find((call) =>
call.url.includes("app.blackbox.ai/api/check-subscription")
);
const museSparkCall = calls.find((call) => call.url.includes("meta.ai/api/graphql"));
// Grok goes through tlsFetchGrok (TLS override), not globalThis.fetch.
assert.ok(grokTlsCall, "grok TLS override was called");
assert.ok(grokTlsCall!.url.includes("grok.com/rest/app-chat/conversations/new"));
assert.equal(
(grokTlsCall!.options.headers as Record<string, string>)["Cookie"],
"sso=grok-cookie"
);
const grokBody = JSON.parse(String(grokTlsCall!.options.body || "{}"));
assert.equal(grokBody.modeId, "fast");
assert.equal("modelName" in grokBody, false);
assert.equal("modelMode" in grokBody, false);
// Perplexity goes through tlsFetchPerplexity (TLS override), not globalThis.fetch.
// options.headers is a plain object; the validator sets Cookie from the session token.
assert.ok(pplxTlsCall, "perplexity TLS override was called");
assert.ok(pplxTlsCall!.url.includes("perplexity.ai/rest/sse/perplexity_ask"));
assert.equal(
(pplxTlsCall!.options.headers as Record<string, string>)["Cookie"],
"__Secure-next-auth.session-token=pplx-cookie"
);
assert.equal(blackboxSessionCall?.init.headers.Cookie, "__Secure-authjs.session-token=bb-cookie");
assert.equal(
blackboxSubscriptionCall?.init.headers.Cookie,
"__Secure-authjs.session-token=bb-cookie"
);
assert.equal(museSparkCall?.init.headers.Cookie, "abra_sess=meta-cookie");
assert.equal(museSparkCall?.init.headers["X-FB-Friendly-Name"], "useEctoSendMessageSubscription");
});
test("web-cookie provider validators surface auth and subscription failures", async () => {
// Perplexity uses tlsFetchPerplexity (TLS-impersonating client). Return 403 to simulate
// an invalid session cookie so the validator emits the expected error message.
__setPplxTlsFetchOverride(async () => {
return { status: 403, headers: new Headers(), text: null, body: null };
});
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
if (target.includes("grok.com/rest/app-chat/conversations/new")) {
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
}
if (target.includes("app.blackbox.ai/api/auth/session")) {
const cookie = (init.headers as Record<string, string>)?.Cookie || "";
if (cookie.includes("expired-cookie")) {
return new Response("null", { status: 200 });
}
return new Response(
JSON.stringify({
user: { id: "bb-user-2", email: "free@example.com" },
}),
{ status: 200 }
);
}
if (target.includes("app.blackbox.ai/api/check-subscription")) {
return new Response(
JSON.stringify({
hasActiveSubscription: false,
isTrialSubscription: false,
previouslySubscribed: true,
plan: "free",
}),
{ status: 200 }
);
}
if (target.includes("meta.ai/api/graphql")) {
return new Response(metaAiSseText("Authentication required to send messages", "ERROR"), {
status: 200,
headers: { "Content-Type": "text/event-stream" },
});
}
throw new Error(`unexpected fetch: ${target}`);
};
const grok = await validateProviderApiKey({ provider: "grok-web", apiKey: "grok-cookie" });
const perplexity = await validateProviderApiKey({
provider: "perplexity-web",
apiKey: "pplx-cookie",
});
const blackboxExpired = await validateProviderApiKey({
provider: "blackbox-web",
apiKey: "expired-cookie",
});
const blackboxNoSubscription = await validateProviderApiKey({
provider: "blackbox-web",
apiKey: "free-account-cookie",
});
const museSpark = await validateProviderApiKey({
provider: "muse-spark-web",
apiKey: "meta-cookie",
});
assert.match(grok.error || "", /Invalid SSO cookie/i);
assert.match(perplexity.error || "", /Invalid Perplexity session cookie/i);
assert.match(blackboxExpired.error || "", /Invalid Blackbox session cookie/i);
assert.match(blackboxNoSubscription.error || "", /no active paid subscription/i);
assert.match(museSpark.error || "", /Invalid Meta AI session cookie/i);
});
test("grok-web validator: full DevTools cookie blob is parsed for the sso value", async () => {
let capturedCookie = "";
__setGrokTlsFetchOverride(async (_url, options) => {
capturedCookie = ((options.headers as Record<string, string>) || {}).Cookie || "";
return { status: 200, headers: new Headers(), text: null, body: null };
});
const blob = "i18nextLng=en; stblid=foo; __cf_bm=bar; sso=eyJTARGET.abc.def; cf_clearance=baz;";
const result = await validateProviderApiKey({ provider: "grok-web", apiKey: blob });
assert.equal(result.valid, true);
assert.equal(capturedCookie, "sso=eyJTARGET.abc.def");
});
test("grok-web validator: empty/missing sso in input returns 'Missing sso cookie'", async () => {
__setGrokTlsFetchOverride(async () => {
throw new Error("validator should short-circuit before fetching");
});
const result = await validateProviderApiKey({
provider: "grok-web",
apiKey: "foo=1; bar=2;",
});
assert.equal(result.valid, false);
assert.match(result.error || "", /Missing sso cookie/i);
});
test("grok-web validator: non-auth 403 is reported as failure with upstream body, not silently passed", async () => {
__setGrokTlsFetchOverride(async () => {
return {
status: 403,
headers: new Headers(),
text: JSON.stringify({ error: { code: 7, message: "Model is not found", details: [] } }),
body: null,
};
});
const result = await validateProviderApiKey({ provider: "grok-web", apiKey: "good-cookie" });
assert.equal(result.valid, false);
assert.match(result.error || "", /Grok rejected validation \(403\)/);
assert.match(result.error || "", /Model is not found/);
});
test("grok-web validator: generic non-auth 403 maps to IP-reputation guidance, not 'invalid cookie' (#3474)", async () => {
__setGrokTlsFetchOverride(async () => {
return { status: 403, headers: new Headers(), text: "Forbidden", body: null };
});
const result = await validateProviderApiKey({ provider: "grok-web", apiKey: "any-cookie" });
assert.equal(result.valid, false);
// A bare/non-auth 403 from grok.com is almost always an anti-bot/IP-reputation
// block — the cookie itself is likely fine. The message must point the user to a
// residential IP or proxy, NOT tell them the cookie is invalid.
assert.match(result.error || "", /residential IP|proxy/i);
assert.doesNotMatch(result.error || "", /invalid SSO cookie/i);
});
test("grok-web validator: anti-bot 'Request rejected' 403 maps to IP-reputation guidance (#3474)", async () => {
__setGrokTlsFetchOverride(async () => {
return {
status: 403,
headers: new Headers(),
text: "Request rejected by anti-bot rules.",
body: null,
};
});
const result = await validateProviderApiKey({ provider: "grok-web", apiKey: "good-cookie" });
assert.equal(result.valid, false);
assert.match(result.error || "", /residential IP|proxy/i);
// Cookie may be fine — must not claim it is invalid/expired.
assert.doesNotMatch(result.error || "", /invalid SSO cookie|expired/i);
});
test("grok-web validator: Cloudflare challenge returned with a 403 status maps to IP guidance (#3474)", async () => {
__setGrokTlsFetchOverride(async () => {
return {
status: 403,
headers: new Headers(),
text: "<html><title>Just a moment...</title><script>window._cf_chl_opt</script></html>",
body: null,
};
});
const result = await validateProviderApiKey({ provider: "grok-web", apiKey: "sso=abc" });
assert.equal(result.valid, false);
assert.match(result.error || "", /residential IP|proxy/i);
});
test("grok-web validator: IP-reputation 403 message does not leak a stack/raw error (Hard Rule #12) (#3474)", async () => {
__setGrokTlsFetchOverride(async () => {
return {
status: 403,
headers: new Headers(),
text: "Request rejected by anti-bot rules.\n at GrokWeb.validate (/app/secret/path.ts:42:13)",
body: null,
};
});
const result = await validateProviderApiKey({ provider: "grok-web", apiKey: "good-cookie" });
assert.equal(result.valid, false);
assert.match(result.error || "", /residential IP|proxy/i);
assert.ok(!(result.error || "").includes("at GrokWeb.validate"));
assert.ok(!(result.error || "").includes("/app/secret/path.ts"));
assert.ok(!(result.error || "").includes(" at "));
});
test("grok-web validator: structured non-auth 403 (resource error) still surfaces upstream body for maintainers (#3474)", async () => {
// Distinct from an anti-bot block: a genuine upstream API error (e.g. the probe
// model was renamed) carries a structured error.message and must NOT be masked
// by the IP-reputation guidance — the maintainer needs to see the real cause.
__setGrokTlsFetchOverride(async () => {
return {
status: 403,
headers: new Headers(),
text: JSON.stringify({ error: { code: 7, message: "Model is not found", details: [] } }),
body: null,
};
});
const result = await validateProviderApiKey({ provider: "grok-web", apiKey: "good-cookie" });
assert.equal(result.valid, false);
assert.match(result.error || "", /Grok rejected validation \(403\)/);
assert.match(result.error || "", /Model is not found/);
assert.doesNotMatch(result.error || "", /residential IP|proxy/i);
});
test("grok-web validator: auth-shaped 401 keeps the re-paste/re-authenticate guidance (no regression) (#3474)", async () => {
__setGrokTlsFetchOverride(async () => {
return { status: 401, headers: new Headers(), text: "Unauthorized", body: null };
});
const result = await validateProviderApiKey({ provider: "grok-web", apiKey: "expired-cookie" });
assert.equal(result.valid, false);
assert.match(result.error || "", /Invalid SSO cookie|re-paste/i);
assert.doesNotMatch(result.error || "", /residential IP|proxy/i);
});
test("grok-web validator: 403 with credential-rejection body is treated as auth-failed", async () => {
__setGrokTlsFetchOverride(async () => {
return {
status: 403,
headers: new Headers(),
text: JSON.stringify({
error: {
code: 16,
message: "Failed to look up session ID. [WKE=unauthenticated:invalid-credentials]",
details: [],
},
}),
body: null,
};
});
const result = await validateProviderApiKey({ provider: "grok-web", apiKey: "bad-cookie" });
assert.equal(result.valid, false);
assert.match(result.error || "", /Invalid SSO cookie/i);
});
test("grok-web validator: TLS client unavailable surfaces actionable error", async () => {
__setGrokTlsFetchOverride(async () => {
const { TlsClientUnavailableError } = await import("../../open-sse/services/grokTlsClient.ts");
throw new TlsClientUnavailableError("native binary not found");
});
const result = await validateProviderApiKey({ provider: "grok-web", apiKey: "sso=abc" });
assert.equal(result.valid, false);
assert.match(result.error || "", /TLS impersonation client unavailable/i);
assert.match(result.error || "", /native binary not found/i);
});
test("grok-web validator: Cloudflare challenge page is detected and reported", async () => {
__setGrokTlsFetchOverride(async () => {
return {
status: 200,
headers: new Headers(),
text: "<html><title>Just a moment...</title><script>window._cf_chl_opt</script></html>",
body: null,
};
});
const result = await validateProviderApiKey({ provider: "grok-web", apiKey: "sso=abc" });
assert.equal(result.valid, false);
assert.match(result.error || "", /Cloudflare anti-bot/i);
});
// ─── chatgpt-web validator ──────────────────────────────────────────────────
// Mocks the TLS-impersonating fetch so unit tests don't need the native binding.
const { __setTlsFetchOverrideForTesting } =
await import("../../open-sse/services/chatgptTlsClient.ts");
function makeTlsResponse(status: number, body: string, headers: Record<string, string> = {}): any {
const h = new Headers();
for (const [k, v] of Object.entries(headers)) h.set(k, v);
return { status, headers: h, text: body, body: null };
}
test.afterEach(() => {
__setTlsFetchOverrideForTesting(null);
});
test("chatgpt-web validator: accepts a valid session response with accessToken", async () => {
let captured: { url: string; opts: any } | null = null;
__setTlsFetchOverrideForTesting(async (url, opts) => {
captured = { url, opts };
return makeTlsResponse(
200,
JSON.stringify({ accessToken: "tok-abc", expires: "2030-01-01T00:00:00Z" }),
{ "content-type": "application/json" }
);
});
const result = await validateProviderApiKey({
provider: "chatgpt-web",
apiKey: "__Secure-next-auth.session-token=eyJSESSION",
});
assert.equal(result.valid, true);
assert.equal(captured?.url, "https://chatgpt.com/api/auth/session");
assert.equal(
(captured?.opts.headers as Record<string, string>).Cookie,
"__Secure-next-auth.session-token=eyJSESSION"
);
});
test("chatgpt-web validator: prepends session-token name to bare values", async () => {
let capturedCookie = "";
__setTlsFetchOverrideForTesting(async (_url, opts) => {
capturedCookie = (opts.headers as Record<string, string>).Cookie || "";
return makeTlsResponse(200, JSON.stringify({ accessToken: "tok" }), {
"content-type": "application/json",
});
});
await validateProviderApiKey({ provider: "chatgpt-web", apiKey: "eyJBARE" });
assert.equal(capturedCookie, "__Secure-next-auth.session-token=eyJBARE");
});
test("chatgpt-web validator: passes full DevTools cookie blob through verbatim", async () => {
let capturedCookie = "";
__setTlsFetchOverrideForTesting(async (_url, opts) => {
capturedCookie = (opts.headers as Record<string, string>).Cookie || "";
return makeTlsResponse(200, JSON.stringify({ accessToken: "tok" }), {
"content-type": "application/json",
});
});
const blob =
"Cookie: oai-did=foo; __Secure-next-auth.session-token.0=eyJchunk0; __Secure-next-auth.session-token.1=eyJchunk1; cf_clearance=cf123;";
await validateProviderApiKey({ provider: "chatgpt-web", apiKey: blob });
assert.equal(
capturedCookie,
"oai-did=foo; __Secure-next-auth.session-token.0=eyJchunk0; __Secure-next-auth.session-token.1=eyJchunk1; cf_clearance=cf123;"
);
});
test("chatgpt-web validator: 401 without cf-mitigated → invalid session cookie", async () => {
__setTlsFetchOverrideForTesting(async () =>
makeTlsResponse(401, JSON.stringify({ error: "unauthorized" }), {
"content-type": "application/json",
})
);
const result = await validateProviderApiKey({
provider: "chatgpt-web",
apiKey: "stale-token",
});
assert.equal(result.valid, false);
assert.match(result.error || "", /Invalid ChatGPT session cookie/i);
});
test("chatgpt-web validator: 403 with cf-mitigated header → Cloudflare hint", async () => {
__setTlsFetchOverrideForTesting(async () =>
makeTlsResponse(403, "<html>Just a moment...</html>", {
"content-type": "text/html",
"cf-mitigated": "challenge",
})
);
const result = await validateProviderApiKey({
provider: "chatgpt-web",
apiKey: "good-but-no-cf-cookies",
});
assert.equal(result.valid, false);
assert.match(result.error || "", /Cloudflare blocked the validator/i);
});
test("chatgpt-web validator: 200 without accessToken → session expired", async () => {
__setTlsFetchOverrideForTesting(async () =>
makeTlsResponse(200, JSON.stringify({}), { "content-type": "application/json" })
);
const result = await validateProviderApiKey({
provider: "chatgpt-web",
apiKey: "expired-token",
});
assert.equal(result.valid, false);
assert.match(result.error || "", /session expired/i);
});
test("chatgpt-web validator: 5xx → ChatGPT unavailable", async () => {
__setTlsFetchOverrideForTesting(async () =>
makeTlsResponse(503, "service unavailable", { "content-type": "text/plain" })
);
const result = await validateProviderApiKey({
provider: "chatgpt-web",
apiKey: "any-token",
});
assert.equal(result.valid, false);
assert.match(result.error || "", /ChatGPT unavailable \(503\)/);
});
test("chatgpt-web validator: 200 non-JSON content-type surfaces a cookie hint", async () => {
__setTlsFetchOverrideForTesting(async () =>
makeTlsResponse(200, "<html>blocked</html>", {
"content-type": "text/html",
"cf-ray": "ray-123",
})
);
const result = await validateProviderApiKey({
provider: "chatgpt-web",
apiKey: "any-token",
});
assert.equal(result.valid, false);
assert.match(result.error || "", /non-JSON.*text\/html.*cf-ray=ray-123/i);
});
test("chatgpt-web validator: TlsClientUnavailableError surfaces a clear message", async () => {
const { TlsClientUnavailableError } = await import("../../open-sse/services/chatgptTlsClient.ts");
__setTlsFetchOverrideForTesting(async () => {
throw new TlsClientUnavailableError("native binding failed to load");
});
const result = await validateProviderApiKey({
provider: "chatgpt-web",
apiKey: "any-token",
});
assert.equal(result.valid, false);
assert.match(result.error || "", /chatgpt-web requires this/i);
});
test("search provider validators cover success, client errors, server errors and custom user agent injection", async () => {
const calls = [];
globalThis.fetch = async (url, init = {}) => {
calls.push({ url: String(url), init });
const target = String(url);
if (target.match(/search\.brave\.com/i)) {
return new Response(JSON.stringify({ results: [] }), { status: 200 });
}
if (target.match(/api\.exa\.ai/i)) {
return new Response(JSON.stringify({ error: "bad key" }), { status: 403 });
}
if (target.match(/api\.tavily\.com/i)) {
return new Response(JSON.stringify({ error: "server" }), { status: 503 });
}
if (target.match(/api\.perplexity\.ai/i)) {
throw new Error("perplexity offline");
}
throw new Error(`unexpected fetch: ${target}`);
};
const brave = await validateProviderApiKey({
provider: "brave-search",
apiKey: "brave-key",
providerSpecificData: { customUserAgent: "SearchSuite/1.0" },
});
const exa = await validateProviderApiKey({ provider: "exa-search", apiKey: "exa-key" });
const tavily = await validateProviderApiKey({ provider: "tavily-search", apiKey: "tv-key" });
const perplexity = await validateProviderApiKey({
provider: "perplexity-search",
apiKey: "px-key",
});
assert.equal(brave.valid, true);
assert.equal(exa.error, "Invalid API key");
assert.equal(tavily.error, "Validation failed: 503");
assert.equal(perplexity.error, "perplexity offline");
assert.equal(calls[0].init.headers["User-Agent"], "SearchSuite/1.0");
});
test("extended search provider validators cover Google PSE, Linkup, SearchAPI, You.com and SearXNG", async () => {
const originalAllowPrivateProviderUrls = process.env.OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS;
process.env.OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS = "true";
const calls = [];
try {
globalThis.fetch = async (url, init = {}) => {
calls.push({ url: String(url), init });
const target = String(url);
if (target.startsWith("https://www.googleapis.com/customsearch/v1")) {
return new Response(JSON.stringify({ items: [] }), { status: 200 });
}
if (target.startsWith("https://api.linkup.so/v1/search")) {
return new Response(JSON.stringify({ error: "bad request" }), { status: 400 });
}
if (target.startsWith("https://www.searchapi.io/api/v1/search")) {
return new Response(JSON.stringify({ organic_results: [] }), { status: 200 });
}
if (target.startsWith("https://ydc-index.io/v1/search")) {
return new Response(JSON.stringify({ results: { web: [] } }), { status: 200 });
}
if (target.startsWith("http://localhost:9999/search")) {
return new Response(JSON.stringify({ results: [] }), { status: 200 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const google = await validateProviderApiKey({
provider: "google-pse-search",
apiKey: "google-key",
providerSpecificData: { cx: "engine-id" },
});
const linkup = await validateProviderApiKey({
provider: "linkup-search",
apiKey: "linkup-key",
});
const searchapi = await validateProviderApiKey({
provider: "searchapi-search",
apiKey: "searchapi-key",
});
const youcom = await validateProviderApiKey({
provider: "youcom-search",
apiKey: "you-key",
});
const searxng = await validateProviderApiKey({
provider: "searxng-search",
providerSpecificData: { baseUrl: "http://localhost:9999/search" },
});
assert.equal(google.valid, true);
assert.equal(linkup.valid, true);
assert.equal(searchapi.valid, true);
assert.equal(youcom.valid, true);
assert.equal(searxng.valid, true);
assert.match(calls[0].url, /cx=engine-id/);
assert.equal(calls[1].init.headers.Authorization, "Bearer linkup-key");
assert.match(calls[2].url, /api_key=searchapi-key/);
assert.equal(calls[3].init.headers["X-API-Key"], "you-key");
} finally {
if (originalAllowPrivateProviderUrls === undefined) {
delete process.env.OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS;
} else {
process.env.OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS = originalAllowPrivateProviderUrls;
}
}
});
test("google PSE validator requires cx", async () => {
const result = await validateProviderApiKey({
provider: "google-pse-search",
apiKey: "google-key",
});
assert.equal(result.valid, false);
assert.equal(result.error, "Programmable Search Engine ID (cx) is required");
});
test("Maritalk validates with Key auth against the models endpoint", async () => {
const calls = [];
globalThis.fetch = async (url, init = {}) => {
calls.push({ url: String(url), init });
assert.equal(String(url), "https://chat.maritaca.ai/api/models");
assert.equal(init.headers.Authorization, "Key maritalk-key");
return new Response(JSON.stringify({ data: [{ id: "sabia-4" }] }), {
status: 200,
});
};
const result = await validateProviderApiKey({
provider: "maritalk",
apiKey: "maritalk-key",
});
assert.equal(result.valid, true);
assert.equal(result.method, "maritalk_models");
assert.equal(calls.length, 1);
});
test("Maritalk falls back to chat probe when the models endpoint is unreachable", async () => {
const calls = [];
globalThis.fetch = async (url, init = {}) => {
calls.push({ url: String(url), init });
if (String(url) === "https://chat.maritaca.ai/api/models") {
return new Response(JSON.stringify({ error: "not found" }), { status: 404 });
}
assert.equal(String(url), "https://chat.maritaca.ai/api/chat/completions");
assert.equal(init.headers.Authorization, "Key maritalk-key");
const body = JSON.parse(String(init.body));
assert.equal(body.model, "sabia-4");
return new Response(JSON.stringify({ choices: [{ message: { content: "ok" } }] }), {
status: 200,
});
};
const result = await validateProviderApiKey({
provider: "maritalk",
apiKey: "maritalk-key",
});
assert.equal(result.valid, true);
assert.equal(calls.length, 2);
});
test("Maritalk treats a rate-limited models probe as valid credentials", async () => {
const calls = [];
globalThis.fetch = async (url, init = {}) => {
calls.push({ url: String(url), init });
assert.equal(String(url), "https://chat.maritaca.ai/api/models");
assert.equal(init.headers.Authorization, "Key maritalk-key");
return new Response(JSON.stringify({ error: "rate limited" }), {
status: 429,
});
};
const result = await validateProviderApiKey({
provider: "maritalk",
apiKey: "maritalk-key",
});
assert.equal(result.valid, true);
assert.equal(result.warning, "Rate limited, but credentials are valid");
assert.equal(calls.length, 1);
});
test("local OpenAI-style providers validate without sending Authorization when apiKey is blank", async () => {
const originalAllowPrivateProviderUrls = process.env.OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS;
process.env.OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS = "true";
const calls = [];
try {
globalThis.fetch = async (url, init = {}) => {
calls.push({ url: String(url), headers: init.headers || {} });
return new Response(JSON.stringify({ data: [{ id: "local-model" }] }), { status: 200 });
};
const lmStudio = await validateProviderApiKey({
provider: "lm-studio",
providerSpecificData: { baseUrl: "http://localhost:1234/v1" },
});
const vllm = await validateProviderApiKey({
provider: "vllm",
providerSpecificData: { baseUrl: "http://localhost:8000/v1" },
});
const lemonade = await validateProviderApiKey({
provider: "lemonade",
providerSpecificData: { baseUrl: "http://localhost:13305/api/v1" },
});
const llamaCpp = await validateProviderApiKey({
provider: "llama-cpp",
providerSpecificData: { baseUrl: "http://127.0.0.1:8080/v1" },
});
assert.equal(lmStudio.valid, true);
assert.equal(vllm.valid, true);
assert.equal(lemonade.valid, true);
assert.equal(llamaCpp.valid, true);
assert.deepEqual(
calls.map((call) => call.url),
[
"http://localhost:1234/v1/models",
"http://localhost:8000/v1/models",
"http://localhost:13305/api/v1/models",
"http://127.0.0.1:8080/v1/models",
]
);
assert.equal(calls[0].headers.Authorization, undefined);
assert.equal(calls[1].headers.Authorization, undefined);
assert.equal(calls[2].headers.Authorization, undefined);
assert.equal(calls[3].headers.Authorization, undefined);
} finally {
if (originalAllowPrivateProviderUrls === undefined) {
delete process.env.OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS;
} else {
process.env.OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS = originalAllowPrivateProviderUrls;
}
}
});
test("OpenAI-compatible validator covers /responses mode and final ping fallback", async () => {
const calls = [];
globalThis.fetch = async (url, init = {}) => {
calls.push({ url: String(url), method: init.method || "GET" });
if (String(url).endsWith("/models")) {
return new Response(JSON.stringify({ error: "no models" }), { status: 500 });
}
if (String(url).endsWith("/responses")) {
return new Response(JSON.stringify({ id: "resp_123" }), { status: 200 });
}
if (String(url) === "https://openai-like.example.com/v1") {
return new Response("ok", { status: 418 });
}
throw new Error(`unexpected fetch: ${url}`);
};
const responsesResult = await validateProviderApiKey({
provider: "openai-compatible-responses",
apiKey: "sk-test",
providerSpecificData: {
baseUrl: "https://openai-like.example.com/v1",
apiType: "responses",
validationModelId: "gpt-test",
},
});
globalThis.fetch = async (url) => {
if (String(url).endsWith("/models")) {
return new Response(JSON.stringify({ error: "server" }), { status: 500 });
}
if (String(url).endsWith("/chat/completions")) {
throw new Error("chat probe offline");
}
return new Response("teapot", { status: 418 });
};
const pingFallback = await validateProviderApiKey({
provider: "openai-compatible-ping-fallback",
apiKey: "sk-test",
providerSpecificData: {
baseUrl: "https://openai-like.example.com/v1",
validationModelId: "gpt-test",
},
});
assert.equal(responsesResult.valid, true);
assert.equal(responsesResult.method, "chat_completions");
assert.deepEqual(
calls.map((call) => call.url),
["https://openai-like.example.com/v1/models", "https://openai-like.example.com/v1/responses"]
);
assert.equal(pingFallback.valid, true);
assert.equal(pingFallback.error, null);
});
test("Anthropic-compatible and Claude Code compatible validators cover direct success and bridge fallbacks", async () => {
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
if (target.match(/anthropic-compatible\.example\.com/i) && init.method === "GET") {
return new Response(JSON.stringify({ data: [] }), { status: 200 });
}
if (target.match(/cc-compatible\.example\.com/i) && init.method === "GET") {
return new Response(JSON.stringify({ error: "bridge unavailable" }), { status: 500 });
}
if (target.match(/cc-compatible\.example\.com/i) && init.method === "POST") {
return new Response(JSON.stringify({ error: "rate limited" }), { status: 429 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const anthropic = await validateProviderApiKey({
provider: "anthropic-compatible-direct",
apiKey: "sk-anthropic",
providerSpecificData: {
baseUrl: "https://anthropic-compatible.example.com/v1/messages",
modelsPath: "/custom-models",
},
});
const ccRateLimited = await validateClaudeCodeCompatibleProvider({
apiKey: "sk-cc",
providerSpecificData: {
baseUrl: "https://cc-compatible.example.com/v1/messages",
validationModelId: "claude-bridge-test",
},
});
globalThis.fetch = async (url, init = {}) => {
if (init.method === "GET") {
return new Response(JSON.stringify({ error: "bridge unavailable" }), { status: 500 });
}
return new Response(JSON.stringify({ error: "bad gateway" }), { status: 502 });
};
const ccFailure = await validateClaudeCodeCompatibleProvider({
apiKey: "sk-cc",
providerSpecificData: {
baseUrl: "https://cc-compatible.example.com/v1/messages",
},
});
assert.equal(anthropic.valid, true);
assert.equal(ccRateLimited.valid, true);
assert.equal(ccRateLimited.method, "cc_bridge_request");
assert.match(ccRateLimited.warning, /Rate limited/i);
assert.equal(ccFailure.valid, false);
assert.equal(ccFailure.error, "Validation failed: 502");
});
test("Claude Code compatible validator rejects missing base URL and bridge auth failures", async () => {
const missingBase = await validateClaudeCodeCompatibleProvider({
apiKey: "sk-cc",
providerSpecificData: {},
});
globalThis.fetch = async (url, init = {}) => {
if (init.method === "GET") {
throw new Error("models offline");
}
return new Response(JSON.stringify({ error: "forbidden" }), { status: 403 });
};
const invalidKey = await validateClaudeCodeCompatibleProvider({
apiKey: "sk-cc",
providerSpecificData: {
baseUrl: "https://cc-compatible.example.com/v1/messages",
},
});
assert.equal(missingBase.error, "No base URL configured for CC Compatible provider");
assert.equal(invalidKey.error, "Invalid API key");
});
test("registry providers cover remaining OpenAI-like and Claude-like validation branches", async () => {
const calls = [];
globalThis.fetch = async (url, init = {}) => {
calls.push({ url: String(url), method: init.method || "GET", headers: init.headers || {} });
const target = String(url);
if (target === "https://api.openai.com/v1/models") {
return new Response(JSON.stringify({ data: [{ id: "gpt-4o-mini" }] }), { status: 200 });
}
if (target === "https://api.anthropic.com/v1/messages?beta=true") {
return new Response(JSON.stringify({ id: "msg_123" }), { status: 200 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const openaiModels = await validateProviderApiKey({ provider: "openai", apiKey: "sk-openai" });
const claudeSuccess = await validateProviderApiKey({ provider: "claude", apiKey: "sk-claude" });
globalThis.fetch = async (url) => {
if (String(url) === "https://api.openai.com/v1/models") {
return new Response(JSON.stringify({ error: "server" }), { status: 500 });
}
if (String(url) === "https://api.openai.com/v1/chat/completions") {
return new Response(JSON.stringify({ error: "method not allowed" }), { status: 405 });
}
throw new Error(`unexpected fetch: ${url}`);
};
const openaiUnsupported = await validateProviderApiKey({
provider: "openai",
apiKey: "sk-openai",
});
globalThis.fetch = async (url) => {
if (String(url) === "https://api.openai.com/v1/models") {
return new Response(JSON.stringify({ error: "server" }), { status: 500 });
}
if (String(url) === "https://api.openai.com/v1/chat/completions") {
return new Response(JSON.stringify({ error: "unprocessable" }), { status: 422 });
}
throw new Error(`unexpected fetch: ${url}`);
};
const openaiInference = await validateProviderApiKey({ provider: "openai", apiKey: "sk-openai" });
globalThis.fetch = async (url) => {
if (String(url) === "https://api.openai.com/v1/models") {
return new Response(JSON.stringify({ error: "server" }), { status: 500 });
}
if (String(url) === "https://api.openai.com/v1/chat/completions") {
return new Response(JSON.stringify({ error: "server" }), { status: 502 });
}
throw new Error(`unexpected fetch: ${url}`);
};
const openaiUnavailable = await validateProviderApiKey({
provider: "openai",
apiKey: "sk-openai",
});
globalThis.fetch = async () =>
new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
const claudeInvalid = await validateProviderApiKey({ provider: "claude", apiKey: "sk-claude" });
globalThis.fetch = async () => {
throw new Error("anthropic offline");
};
const claudeOffline = await validateProviderApiKey({ provider: "claude", apiKey: "sk-claude" });
assert.equal(openaiModels.valid, true);
assert.equal(openaiModels.error, null);
assert.equal(claudeSuccess.valid, true);
assert.equal(openaiUnsupported.error, "Provider validation endpoint not supported");
assert.equal(openaiInference.valid, true);
assert.equal(openaiInference.error, null);
assert.equal(openaiUnavailable.error, "Provider unavailable (502)");
assert.equal(claudeInvalid.error, "Invalid API key");
assert.equal(claudeOffline.error, "anthropic offline");
assert.equal(calls[1].headers["x-api-key"], "sk-claude");
});
test("specialty validators cover remaining status branches for Deepgram, AssemblyAI, ElevenLabs, Inworld, Bailian and LongCat", async () => {
globalThis.fetch = async (url) => {
const target = String(url);
if (target.match(/deepgram/i)) {
return new Response(JSON.stringify({ error: "server" }), { status: 500 });
}
if (target.match(/assemblyai/i)) {
return new Response(JSON.stringify({ transcripts: [] }), { status: 200 });
}
if (target.match(/elevenlabs/i)) {
return new Response(JSON.stringify({ error: "forbidden" }), { status: 403 });
}
if (target.match(/inworld/i)) {
throw new Error("inworld offline");
}
if (target.match(/dashscope\.aliyuncs\.com/i)) {
return new Response(JSON.stringify({ error: "server" }), { status: 500 });
}
if (target.match(/longcat/i)) {
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const deepgram = await validateProviderApiKey({ provider: "deepgram", apiKey: "dg-key" });
const assembly = await validateProviderApiKey({ provider: "assemblyai", apiKey: "aa-key" });
const eleven = await validateProviderApiKey({ provider: "elevenlabs", apiKey: "el-key" });
const inworld = await validateProviderApiKey({ provider: "inworld", apiKey: "iw-key" });
const bailian = await validateProviderApiKey({
provider: "bailian-coding-plan",
apiKey: "bailian-key",
providerSpecificData: {
baseUrl: "https://coding-intl.dashscope.aliyuncs.com/apps/anthropic/v1/messages",
},
});
const longcatInvalid = await validateProviderApiKey({ provider: "longcat", apiKey: "lc-key" });
globalThis.fetch = async (url) => {
if (String(url).match(/elevenlabs/i)) {
throw new Error("elevenlabs offline");
}
if (String(url).match(/longcat/i)) {
return new Response(JSON.stringify({ error: "unprocessable" }), { status: 422 });
}
throw new Error(`unexpected fetch: ${url}`);
};
const elevenOffline = await validateProviderApiKey({ provider: "elevenlabs", apiKey: "el-key" });
const longcatValid = await validateProviderApiKey({ provider: "longcat", apiKey: "lc-key" });
assert.equal(deepgram.error, "Validation failed: 500");
assert.equal(assembly.valid, true);
assert.equal(eleven.error, "Invalid API key");
assert.equal(inworld.error, "inworld offline");
assert.equal(bailian.error, "Validation failed: 500");
assert.equal(longcatInvalid.error, "Invalid API key");
assert.equal(elevenOffline.error, "elevenlabs offline");
assert.equal(longcatValid.valid, true);
});
test("specialty validators cover Heroku, Databricks, Snowflake and GigaChat success paths", async () => {
const seen = [];
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
seen.push({ url: target, headers: init.headers || {} });
if (target === "https://ngw.devices.sberbank.ru:9443/api/v2/oauth") {
assert.equal(init.headers.Authorization, "Basic gigachat-basic-creds");
return new Response(
JSON.stringify({
tok: "gigachat-access-token",
exp: Date.now() + 60 * 60 * 1000,
}),
{ status: 200 }
);
}
if (target === "https://us.inference.heroku.com/v1/chat/completions") {
assert.equal(init.headers.Authorization, "Bearer heroku-key");
return new Response(JSON.stringify({ error: "bad request" }), { status: 400 });
}
if (
target ===
"https://adb-1234567890123456.7.azuredatabricks.net/serving-endpoints/chat/completions"
) {
assert.equal(init.headers.Authorization, "Bearer databricks-key");
return new Response(JSON.stringify({ error: "unprocessable" }), { status: 422 });
}
if (target === "https://account.snowflakecomputing.com/api/v2/cortex/inference:complete") {
assert.equal(init.headers.Authorization, "Bearer snowflake-token");
assert.equal(
init.headers["X-Snowflake-Authorization-Token-Type"],
"PROGRAMMATIC_ACCESS_TOKEN"
);
return new Response(JSON.stringify({ error: "bad request" }), { status: 400 });
}
if (target === "https://gigachat.devices.sberbank.ru/api/v1/chat/completions") {
assert.equal(init.headers.Authorization, "Bearer gigachat-access-token");
return new Response(JSON.stringify({ error: "bad request" }), { status: 400 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const heroku = await validateProviderApiKey({
provider: "heroku",
apiKey: "heroku-key",
providerSpecificData: { baseUrl: "https://us.inference.heroku.com" },
});
const databricks = await validateProviderApiKey({
provider: "databricks",
apiKey: "databricks-key",
providerSpecificData: {
baseUrl: "https://adb-1234567890123456.7.azuredatabricks.net/serving-endpoints",
},
});
const snowflake = await validateProviderApiKey({
provider: "snowflake",
apiKey: "pat/snowflake-token",
providerSpecificData: { baseUrl: "https://account.snowflakecomputing.com" },
});
const gigachat = await validateProviderApiKey({
provider: "gigachat",
apiKey: "gigachat-basic-creds",
});
assert.equal(heroku.valid, true);
assert.equal(databricks.valid, true);
assert.equal(snowflake.valid, true);
assert.equal(gigachat.valid, true);
assert.equal(
seen.some((call) => call.url === "https://ngw.devices.sberbank.ru:9443/api/v2/oauth"),
true
);
});
test("specialty validators surface missing base URLs and invalid auth for Heroku, Databricks, Snowflake and GigaChat", async () => {
const missingHerokuBase = await validateProviderApiKey({
provider: "heroku",
apiKey: "heroku-key",
providerSpecificData: {},
});
const missingDatabricksBase = await validateProviderApiKey({
provider: "databricks",
apiKey: "databricks-key",
providerSpecificData: {},
});
const missingSnowflakeBase = await validateProviderApiKey({
provider: "snowflake",
apiKey: "snowflake-key",
providerSpecificData: {},
});
globalThis.fetch = async (url) => {
const target = String(url);
if (target === "https://ngw.devices.sberbank.ru:9443/api/v2/oauth") {
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
}
if (target === "https://us.inference.heroku.com/v1/chat/completions") {
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
}
if (
target ===
"https://adb-1234567890123456.7.azuredatabricks.net/serving-endpoints/chat/completions"
) {
return new Response(JSON.stringify({ error: "forbidden" }), { status: 403 });
}
if (target === "https://account.snowflakecomputing.com/api/v2/cortex/inference:complete") {
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const herokuInvalid = await validateProviderApiKey({
provider: "heroku",
apiKey: "heroku-key",
providerSpecificData: { baseUrl: "https://us.inference.heroku.com" },
});
const databricksInvalid = await validateProviderApiKey({
provider: "databricks",
apiKey: "databricks-key",
providerSpecificData: {
baseUrl: "https://adb-1234567890123456.7.azuredatabricks.net/serving-endpoints",
},
});
const snowflakeInvalid = await validateProviderApiKey({
provider: "snowflake",
apiKey: "snowflake-key",
providerSpecificData: { baseUrl: "https://account.snowflakecomputing.com" },
});
const gigachatInvalid = await validateProviderApiKey({
provider: "gigachat",
apiKey: "gigachat-basic-creds-invalid",
});
assert.equal(missingHerokuBase.error, "Missing base URL");
assert.equal(missingDatabricksBase.error, "Missing base URL");
assert.equal(missingSnowflakeBase.error, "Missing base URL");
assert.equal(herokuInvalid.error, "Invalid API key");
assert.equal(databricksInvalid.error, "Invalid API key");
assert.equal(snowflakeInvalid.error, "Invalid API key");
assert.equal(gigachatInvalid.error, "Invalid API key");
});
test("specialty validator accepts DataRobot gateway and deployment credentials", async () => {
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
if (target === "https://app.datarobot.com/genai/llmgw/catalog/") {
assert.equal((init.headers as Record<string, string>).Authorization, "Bearer dr-key");
return new Response(
JSON.stringify({
data: [{ model: "azure/gpt-5-mini-2025-08-07", isActive: true }],
}),
{ status: 200 }
);
}
if (
target ===
"https://app.datarobot.com/api/v2/deployments/65f5b2b7c8f8c4b257e0d123/chat/completions"
) {
assert.equal((init.headers as Record<string, string>).Authorization, "Bearer dr-deploy-key");
const body = JSON.parse(String(init.body));
assert.equal(body.model, "datarobot-deployed-llm");
return new Response(JSON.stringify({ error: "bad request" }), { status: 400 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const gateway = await validateProviderApiKey({
provider: "datarobot",
apiKey: "dr-key",
});
const deployment = await validateProviderApiKey({
provider: "datarobot",
apiKey: "dr-deploy-key",
providerSpecificData: {
baseUrl: "https://app.datarobot.com/api/v2/deployments/65f5b2b7c8f8c4b257e0d123",
},
});
assert.equal(gateway.valid, true);
assert.equal(deployment.valid, true);
});
test("specialty validator rejects invalid DataRobot credentials", async () => {
globalThis.fetch = async (url) => {
const target = String(url);
if (target === "https://app.datarobot.com/genai/llmgw/catalog/") {
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
}
if (
target ===
"https://app.datarobot.com/api/v2/deployments/65f5b2b7c8f8c4b257e0d123/chat/completions"
) {
return new Response(JSON.stringify({ error: "forbidden" }), { status: 403 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const gateway = await validateProviderApiKey({
provider: "datarobot",
apiKey: "dr-key",
});
const deployment = await validateProviderApiKey({
provider: "datarobot",
apiKey: "dr-deploy-key",
providerSpecificData: {
baseUrl: "https://app.datarobot.com/api/v2/deployments/65f5b2b7c8f8c4b257e0d123",
},
});
assert.equal(gateway.error, "Invalid API key");
assert.equal(deployment.error, "Invalid API key");
});
test("specialty validators accept watsonx, OCI and SAP enterprise gateways", async () => {
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
if (target === "https://ca-tor.ml.cloud.ibm.com/ml/gateway/v1/models") {
assert.equal((init.headers as Record<string, string>).Authorization, "Bearer watsonx-key");
return new Response(JSON.stringify({ data: [] }), { status: 200 });
}
if (
target === "https://inference.generativeai.us-chicago-1.oci.oraclecloud.com/openai/v1/models"
) {
const headers = init.headers as Record<string, string>;
assert.equal(headers.Authorization, "Bearer oci-key");
assert.equal(headers["OpenAI-Project"], "ocid1.generativeaiproject.oc1.us-chicago-1.demo");
return new Response(JSON.stringify({ data: [] }), { status: 200 });
}
if (target === "https://sap.example.com/v2/lm/scenarios/foundation-models/models") {
const headers = init.headers as Record<string, string>;
assert.equal(headers.Authorization, "Bearer sap-key");
assert.equal(headers["AI-Resource-Group"], "shared");
return new Response(JSON.stringify({ resources: [] }), { status: 200 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const watsonx = await validateProviderApiKey({
provider: "watsonx",
apiKey: "watsonx-key",
providerSpecificData: { baseUrl: "https://ca-tor.ml.cloud.ibm.com" },
});
const oci = await validateProviderApiKey({
provider: "oci",
apiKey: "oci-key",
providerSpecificData: {
baseUrl: "https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
projectId: "ocid1.generativeaiproject.oc1.us-chicago-1.demo",
},
});
const sap = await validateProviderApiKey({
provider: "sap",
apiKey: "sap-key",
providerSpecificData: {
baseUrl: "https://sap.example.com/v2/lm/deployments/demo-deployment",
resourceGroup: "shared",
},
});
assert.equal(watsonx.valid, true);
assert.equal(watsonx.method, "watsonx_models");
assert.equal(oci.valid, true);
assert.equal(oci.method, "oci_models");
assert.equal(sap.valid, true);
assert.equal(sap.method, "sap_models");
});
test("specialty validator accepts native Bedrock model discovery with a configured region", async () => {
const seenUrls: string[] = [];
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
seenUrls.push(target);
assert.equal((init.headers as Record<string, string>).Authorization, "Bearer bedrock-key");
if (
target === "https://bedrock.eu-west-2.amazonaws.com/foundation-models?byOutputModality=TEXT"
) {
return new Response(
JSON.stringify({
modelSummaries: [
{
modelId: "anthropic.claude-sonnet-4-6",
modelName: "Claude Sonnet 4.6",
providerName: "Anthropic",
inputModalities: ["TEXT", "IMAGE"],
outputModalities: ["TEXT"],
responseStreamingSupported: true,
},
],
}),
{ status: 200 }
);
}
if (
target ===
"https://bedrock.eu-west-2.amazonaws.com/inference-profiles?maxResults=100&typeEquals=SYSTEM_DEFINED"
) {
return new Response(
JSON.stringify({
inferenceProfileSummaries: [
{
inferenceProfileId: "eu.anthropic.claude-sonnet-4-6",
inferenceProfileName: "EU Claude Sonnet 4.6",
models: [
{
modelArn:
"arn:aws:bedrock:eu-west-2::foundation-model/anthropic.claude-sonnet-4-6",
},
],
},
],
}),
{ status: 200 }
);
}
throw new Error("unexpected fetch: " + target);
};
const bedrock = await validateProviderApiKey({
provider: "bedrock",
apiKey: "bedrock-key",
providerSpecificData: { region: "eu-west-2" },
});
assert.equal(bedrock.valid, true);
assert.equal(bedrock.method, "bedrock_native_models");
assert.deepEqual(seenUrls, [
"https://bedrock.eu-west-2.amazonaws.com/foundation-models?byOutputModality=TEXT",
"https://bedrock.eu-west-2.amazonaws.com/inference-profiles?maxResults=100&typeEquals=SYSTEM_DEFINED",
]);
});
test("specialty validator rejects invalid native Bedrock credentials", async () => {
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
if (
target === "https://bedrock.eu-west-2.amazonaws.com/foundation-models?byOutputModality=TEXT"
) {
assert.equal((init.headers as Record<string, string>).Authorization, "Bearer bedrock-key");
return new Response(JSON.stringify({ message: "forbidden" }), { status: 403 });
}
throw new Error("unexpected fetch: " + target);
};
const bedrock = await validateProviderApiKey({
provider: "bedrock",
apiKey: "bedrock-key",
providerSpecificData: { region: "eu-west-2" },
});
assert.equal(bedrock.error, "Invalid API key");
});
test("specialty validators reject invalid watsonx, OCI and SAP credentials", async () => {
globalThis.fetch = async (url) => {
const target = String(url);
if (target === "https://ca-tor.ml.cloud.ibm.com/ml/gateway/v1/models") {
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
}
if (
target === "https://inference.generativeai.us-chicago-1.oci.oraclecloud.com/openai/v1/models"
) {
return new Response(JSON.stringify({ error: "forbidden" }), { status: 403 });
}
if (target === "https://sap.example.com/v2/lm/scenarios/foundation-models/models") {
return new Response(JSON.stringify({ error: "forbidden" }), { status: 403 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const watsonx = await validateProviderApiKey({
provider: "watsonx",
apiKey: "watsonx-key",
providerSpecificData: { baseUrl: "https://ca-tor.ml.cloud.ibm.com" },
});
const oci = await validateProviderApiKey({
provider: "oci",
apiKey: "oci-key",
providerSpecificData: {
baseUrl: "https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
},
});
const sap = await validateProviderApiKey({
provider: "sap",
apiKey: "sap-key",
providerSpecificData: {
baseUrl: "https://sap.example.com/v2/lm/deployments/demo-deployment",
},
});
assert.equal(watsonx.error, "Invalid API key");
assert.equal(oci.error, "Invalid API key");
assert.equal(sap.error, "Invalid API key");
});
test("specialty validator accepts Modal OpenAI-compatible deployments", async () => {
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
if (target === "https://alice--demo.modal.run/v1/models") {
const headers = init.headers as Record<string, string>;
assert.equal(headers.Authorization, "Bearer modal-key");
return new Response(JSON.stringify({ data: [] }), { status: 200 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const modal = await validateProviderApiKey({
provider: "modal",
apiKey: "modal-key",
providerSpecificData: {
baseUrl: "https://alice--demo.modal.run/v1",
},
});
assert.equal(modal.valid, true);
});
test("specialty validator rejects invalid Modal credentials", async () => {
globalThis.fetch = async (url) => {
const target = String(url);
if (target === "https://alice--demo.modal.run/v1/models") {
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const modal = await validateProviderApiKey({
provider: "modal",
apiKey: "modal-key",
providerSpecificData: {
baseUrl: "https://alice--demo.modal.run/v1",
},
});
assert.equal(modal.error, "Invalid API key");
});
test("specialty validator accepts Poe credentials on the current balance endpoint", async () => {
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
if (target === "https://api.poe.com/usage/current_balance") {
const headers = init.headers as Record<string, string>;
assert.equal(headers.Authorization, "Bearer poe-key");
return new Response(JSON.stringify({ current_point_balance: 123456 }), { status: 200 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const poe = await validateProviderApiKey({
provider: "poe",
apiKey: "poe-key",
});
assert.equal(poe.valid, true);
assert.equal(poe.method, "poe_current_balance");
});
test("specialty validator accepts Nous Research credentials on chat completions", async () => {
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
if (target === "https://inference-api.nousresearch.com/v1/chat/completions") {
const headers = init.headers as Record<string, string>;
const body = JSON.parse(String(init.body));
assert.equal(headers.Authorization, "Bearer nous-key");
assert.equal(body.model, "Hermes-4-70B");
return new Response(
JSON.stringify({
id: "chatcmpl-nous",
choices: [{ message: { role: "assistant", content: "ok" } }],
}),
{ status: 200 }
);
}
throw new Error(`unexpected fetch: ${target}`);
};
const nous = await validateProviderApiKey({
provider: "nous-research",
apiKey: "nous-key",
});
assert.equal(nous.valid, true);
assert.equal(nous.method, "nous_chat_completions");
});
test("BytePlus key validation reaches the Ark endpoint instead of 'not supported' (#3877)", async () => {
// #3877: byteplus was in APIKEY_PROVIDERS but never registered in the routing
// registry, so validation returned {unsupported:true} → UI showed "invalid" for any
// key. With the registry entry, a valid ark-... key probes the Ark /models endpoint
// with Bearer auth and validates.
let probedModelsUrl: string | null = null;
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
if (target === "https://ark.ap-southeast.bytepluses.com/api/v3/models") {
probedModelsUrl = target;
const headers = toPlainHeaders(init.headers);
assert.equal(headers.Authorization, "Bearer ark-test-key");
return new Response(JSON.stringify({ data: [{ id: "kimi-k2-thinking" }] }), { status: 200 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const result = await validateProviderApiKey({
provider: "byteplus",
apiKey: "ark-test-key",
});
assert.equal(result.unsupported, undefined, "byteplus must not be 'validation not supported'");
assert.equal(result.valid, true);
assert.equal(probedModelsUrl, "https://ark.ap-southeast.bytepluses.com/api/v3/models");
});
test("specialty validator rejects invalid Nous Research credentials", async () => {
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
if (target === "https://inference-api.nousresearch.com/v1/chat/completions") {
const headers = init.headers as Record<string, string>;
assert.equal(headers.Authorization, "Bearer nous-bad");
return new Response(JSON.stringify({ message: "invalid" }), { status: 401 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const nous = await validateProviderApiKey({
provider: "nous-research",
apiKey: "nous-bad",
});
assert.equal(nous.error, "Invalid API key");
});
test("specialty validator accepts Nous Research key when probe model is rejected (400)", async () => {
// #3881: a valid key whose probe model is rejected (model-not-found / bad request)
// must still validate — the 4xx proves auth was accepted, only the request shape
// was wrong. Mirrors the longcat/nvidia validators.
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
if (target === "https://inference-api.nousresearch.com/v1/chat/completions") {
const headers = init.headers as Record<string, string>;
assert.equal(headers.Authorization, "Bearer nous-key");
return new Response(
JSON.stringify({ error: { message: "model not found", type: "invalid_request_error" } }),
{ status: 400 }
);
}
throw new Error(`unexpected fetch: ${target}`);
};
const nous = await validateProviderApiKey({
provider: "nous-research",
apiKey: "nous-key",
});
assert.equal(nous.valid, true);
assert.equal(nous.method, "nous_chat_completions");
});
test("specialty validator rejects invalid Poe credentials", async () => {
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
if (target === "https://api.poe.com/usage/current_balance") {
const headers = init.headers as Record<string, string>;
assert.equal(headers.Authorization, "Bearer poe-bad");
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const poe = await validateProviderApiKey({
provider: "poe",
apiKey: "poe-bad",
});
assert.equal(poe.error, "Invalid API key");
});
test("specialty validator accepts Clarifai credentials through the OpenAI-compatible models probe", async () => {
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
if (target === "https://api.clarifai.com/v2/ext/openai/v1/models") {
const headers = init.headers as Record<string, string>;
assert.equal(headers.Authorization, "Key clarifai-pat");
return new Response(
JSON.stringify({ data: [{ id: "openai/chat-completion/models/gpt-oss-120b" }] }),
{ status: 200 }
);
}
throw new Error(`unexpected fetch: ${target}`);
};
const clarifai = await validateProviderApiKey({
provider: "clarifai",
apiKey: "clarifai-pat",
});
assert.equal(clarifai.valid, true);
assert.equal(clarifai.method, "clarifai_models");
});
test("specialty validator rejects invalid Clarifai credentials", async () => {
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
if (target === "https://api.clarifai.com/v2/ext/openai/v1/models") {
const headers = init.headers as Record<string, string>;
assert.equal(headers.Authorization, "Key clarifai-bad");
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const clarifai = await validateProviderApiKey({
provider: "clarifai",
apiKey: "clarifai-bad",
});
assert.equal(clarifai.error, "Invalid API key");
});
test("specialty validator accepts Reka credentials through the models probe with dual auth headers", async () => {
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
if (target === "https://api.reka.ai/v1/models") {
const headers = init.headers as Record<string, string>;
assert.equal(headers.Authorization, "Bearer reka-key");
assert.equal(headers["X-Api-Key"], "reka-key");
return new Response(JSON.stringify([{ id: "reka-core" }]), { status: 200 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const reka = await validateProviderApiKey({
provider: "reka",
apiKey: "reka-key",
});
assert.equal(reka.valid, true);
assert.equal(reka.method, "reka_models");
});
test("specialty validator rejects invalid Reka credentials", async () => {
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
if (target === "https://api.reka.ai/v1/models") {
const headers = init.headers as Record<string, string>;
assert.equal(headers.Authorization, "Bearer reka-bad");
assert.equal(headers["X-Api-Key"], "reka-bad");
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const reka = await validateProviderApiKey({
provider: "reka",
apiKey: "reka-bad",
});
assert.equal(reka.error, "Invalid API key");
});
test("specialty validator accepts NLP Cloud credentials on the chatbot endpoint", async () => {
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
if (target === "https://api.nlpcloud.io/v1/gpu/chatdolphin/chatbot") {
const headers = init.headers as Record<string, string>;
const body = JSON.parse(String(init.body));
assert.equal(headers.Authorization, "Token nlpc-key");
assert.equal(body.input, "test");
return new Response(JSON.stringify({ response: "ok" }), { status: 200 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const nlpCloud = await validateProviderApiKey({
provider: "nlpcloud",
apiKey: "nlpc-key",
});
assert.equal(nlpCloud.valid, true);
assert.equal(nlpCloud.method, "nlpcloud_chatbot");
});
test("specialty validator rejects invalid NLP Cloud credentials", async () => {
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
if (target === "https://api.nlpcloud.io/v1/gpu/chatdolphin/chatbot") {
const headers = init.headers as Record<string, string>;
assert.equal(headers.Authorization, "Token nlpc-bad");
return new Response(JSON.stringify({ detail: "forbidden" }), { status: 403 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const nlpCloud = await validateProviderApiKey({
provider: "nlpcloud",
apiKey: "nlpc-bad",
});
assert.equal(nlpCloud.error, "Invalid API key");
});
test("specialty validator accepts Runway credentials on the organization endpoint", async () => {
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
if (target === "https://api.dev.runwayml.com/v1/organization") {
const headers = init.headers as Record<string, string>;
assert.equal(headers.Authorization, "Bearer runway-key");
assert.equal(headers["X-Runway-Version"], "2024-11-06");
return new Response(JSON.stringify({ id: "org_demo" }), { status: 200 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const runway = await validateProviderApiKey({
provider: "runwayml",
apiKey: "runway-key",
});
assert.equal(runway.valid, true);
assert.equal(runway.method, "runway_organization");
});
test("specialty validator rejects invalid Runway credentials", async () => {
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
if (target === "https://api.dev.runwayml.com/v1/organization") {
const headers = init.headers as Record<string, string>;
assert.equal(headers.Authorization, "Bearer runway-bad");
assert.equal(headers["X-Runway-Version"], "2024-11-06");
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const runway = await validateProviderApiKey({
provider: "runwayml",
apiKey: "runway-bad",
});
assert.equal(runway.error, "Invalid API key");
});
test("validateCommandCodeProvider sends Command Code probe URL, headers, and wrapper body", async () => {
const calls: any[] = [];
globalThis.fetch = async (url, init = {}) => {
calls.push({
url: String(url),
method: init.method,
headers: toPlainHeaders(init.headers),
body: JSON.parse(String(init.body)),
});
return new Response("", { status: 400 });
};
const result = await validateCommandCodeProvider({
apiKey: "cc_test_key",
providerSpecificData: { validationModelId: "gpt-5.4-mini" },
});
assert.deepEqual(result, { valid: true, error: null });
assert.equal(calls.length, 1);
assert.equal(calls[0].url, "https://api.commandcode.ai/alpha/generate");
assert.equal(calls[0].method, "POST");
assert.equal(calls[0].headers.Authorization, "Bearer cc_test_key");
assert.equal(calls[0].headers["Content-Type"], "application/json");
assert.equal(calls[0].headers["x-command-code-version"], COMMAND_CODE_VERSION);
assert.equal(calls[0].headers["x-cli-environment"], "external");
assert.equal(calls[0].headers["x-project-slug"], "pi-cc");
assert.equal(calls[0].headers["x-taste-learning"], "false");
assert.equal(calls[0].headers["x-co-flag"], "false");
assert.equal(typeof calls[0].headers["x-session-id"], "string");
assert.equal(calls[0].body.config.environment, "external");
assert.equal(calls[0].body.permissionMode, "standard");
assert.equal(calls[0].body.skills, "");
assert.equal(calls[0].body.params.model, "gpt-5.4-mini");
assert.equal(calls[0].body.params.stream, true);
assert.equal(calls[0].body.params.max_tokens, 1);
});
for (const status of [400, 422, 429]) {
test(`validateCommandCodeProvider accepts ${status} as direct validator auth success`, async () => {
globalThis.fetch = async () => new Response("", { status });
assert.deepEqual(await validateCommandCodeProvider({ apiKey: "cc_test_key" }), {
valid: true,
error: null,
});
});
}
test("validateCommandCodeProvider rejects auth failures and provider outages", async () => {
globalThis.fetch = async () => new Response("unauthorized", { status: 401 });
assert.deepEqual(await validateCommandCodeProvider({ apiKey: "bad" }), {
valid: false,
error: "Invalid API key",
});
globalThis.fetch = async () => new Response("server down", { status: 500 });
assert.deepEqual(await validateCommandCodeProvider({ apiKey: "cc_test_key" }), {
valid: false,
error: "Provider unavailable (500)",
});
});
// ─── claude-web validator ────────────────────────────────────────────────────
const { __setTlsFetchOverrideForTesting: __setClaudeTlsFetchOverride } =
await import("../../open-sse/services/claudeTlsClient.ts");
function makeClaudeTlsResponse(
status: number,
body: string,
headers: Record<string, string> = {}
): any {
const h = new Headers();
for (const [k, v] of Object.entries(headers)) h.set(k, v);
return { status, ok: status >= 200 && status < 300, headers: h, text: body, body: null };
}
test("claude-web validator: 200 from /api/organizations → valid", async () => {
let captured: { url: string; opts: any } | null = null;
__setClaudeTlsFetchOverride(async (url, opts) => {
captured = { url, opts };
return makeClaudeTlsResponse(200, JSON.stringify({ orgs: [] }));
});
const result = await validateProviderApiKey({
provider: "claude-web",
apiKey: "sessionKey=sk-ant-sid02-test-session-key",
});
assert.equal(result.valid, true);
assert.equal(result.error, null);
assert.equal(captured?.url, "https://claude.ai/api/organizations");
assert.match(
(captured?.opts.headers as Record<string, string>).Cookie || "",
/sessionKey=sk-ant-sid02-test-session-key/
);
__setClaudeTlsFetchOverride(null);
});
test("claude-web validator: full cookie blob passes through verbatim", async () => {
let capturedCookie = "";
__setClaudeTlsFetchOverride(async (_url, opts) => {
capturedCookie = (opts.headers as Record<string, string>).Cookie || "";
return makeClaudeTlsResponse(200, JSON.stringify({ orgs: [] }));
});
const blob =
"__cf_bm=abc123; sessionKey=sk-ant-sid02-test; intercom-device-id-lupk8zyo=xyz; __stripe_mid=stripe123";
await validateProviderApiKey({ provider: "claude-web", apiKey: blob });
assert.equal(capturedCookie, blob);
__setClaudeTlsFetchOverride(null);
});
test("claude-web validator: 401 → invalid session cookie", async () => {
__setClaudeTlsFetchOverride(async () =>
makeClaudeTlsResponse(401, JSON.stringify({ error: "unauthorized" }))
);
const result = await validateProviderApiKey({
provider: "claude-web",
apiKey: "sessionKey=expired-key",
});
assert.equal(result.valid, false);
assert.match(result.error || "", /Invalid or expired session cookie/i);
__setClaudeTlsFetchOverride(null);
});
test("claude-web validator: 429 → valid (rate limited means auth passed)", async () => {
__setClaudeTlsFetchOverride(async () =>
makeClaudeTlsResponse(429, JSON.stringify({ error: "rate limited" }))
);
const result = await validateProviderApiKey({
provider: "claude-web",
apiKey: "sessionKey=sk-ant-sid02-good-key",
});
assert.equal(result.valid, true);
__setClaudeTlsFetchOverride(null);
});
test("claude-web validator: 500 → Claude.ai unavailable", async () => {
__setClaudeTlsFetchOverride(async () => makeClaudeTlsResponse(500, "internal server error"));
const result = await validateProviderApiKey({
provider: "claude-web",
apiKey: "sessionKey=sk-ant-sid02-any-key",
});
assert.equal(result.valid, false);
assert.match(result.error || "", /Claude\.ai unavailable \(500\)/i);
__setClaudeTlsFetchOverride(null);
});
test("claude-web validator: TLS client unavailable → clear error", async () => {
const { TlsClientUnavailableError } = await import("../../open-sse/services/claudeTlsClient.ts");
__setClaudeTlsFetchOverride(async () => {
throw new TlsClientUnavailableError("tls-client-node not installed");
});
const result = await validateProviderApiKey({
provider: "claude-web",
apiKey: "sessionKey=sk-ant-sid02-any-key",
});
assert.equal(result.valid, false);
assert.match(result.error || "", /tls-client-node not installed/i);
__setClaudeTlsFetchOverride(null);
});
test("claude-web validator: bare sessionKey value gets prefixed", async () => {
let capturedCookie = "";
__setClaudeTlsFetchOverride(async (_url, opts) => {
capturedCookie = (opts.headers as Record<string, string>).Cookie || "";
return makeClaudeTlsResponse(200, JSON.stringify({ orgs: [] }));
});
await validateProviderApiKey({
provider: "claude-web",
apiKey: "sk-ant-sid02-bare-value",
});
assert.equal(capturedCookie, "sessionKey=sk-ant-sid02-bare-value");
__setClaudeTlsFetchOverride(null);
});
// ─── gemini-web validator ────────────────────────────────────────────────────
test("gemini-web validator: 200 from gemini.google.com → valid", async () => {
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
const headers = init.headers || {};
if (target.includes("gemini.google.com/app")) {
assert.match((headers as Record<string, string>).Cookie || "", /__Secure-1PSID=eyJPSID/);
return new Response("ok", { status: 200 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const result = await validateProviderApiKey({
provider: "gemini-web",
apiKey: "__Secure-1PSID=eyJPSID",
});
assert.equal(result.valid, true);
assert.equal(result.error, null);
});
test("gemini-web validator: bare value gets __Secure-1PSID prefix", async () => {
let capturedCookie = "";
globalThis.fetch = async (url, init = {}) => {
if (String(url).includes("gemini.google.com")) {
capturedCookie = ((init.headers as Record<string, string>) || {}).Cookie || "";
return new Response("ok", { status: 200 });
}
throw new Error(`unexpected fetch: ${String(url)}`);
};
await validateProviderApiKey({ provider: "gemini-web", apiKey: "eyJbarevalue" });
assert.equal(capturedCookie, "__Secure-1PSID=eyJbarevalue");
});
test("gemini-web validator: 401 → invalid cookie", async () => {
globalThis.fetch = async () => new Response("unauthorized", { status: 401 });
const result = await validateProviderApiKey({
provider: "gemini-web",
apiKey: "__Secure-1PSID=expired",
});
assert.equal(result.valid, false);
assert.match(result.error || "", /Invalid or expired __Secure-1PSID/i);
});
test("gemini-web validator: 500 → unavailable", async () => {
globalThis.fetch = async () => new Response("down", { status: 500 });
const result = await validateProviderApiKey({
provider: "gemini-web",
apiKey: "__Secure-1PSID=eyJany",
});
assert.equal(result.valid, false);
assert.match(result.error || "", /Gemini validation failed \(500\)/i);
});
// ─── copilot-web validator ───────────────────────────────────────────────────
test("copilot-web validator: valid access_token → 200", async () => {
globalThis.fetch = async (url, init = {}) => {
const target = String(url);
if (target.includes("copilot.microsoft.com/c/api/conversations")) {
assert.match(
((init.headers as Record<string, string>) || {}).Authorization || "",
/Bearer eyJhbGci/
);
return new Response(JSON.stringify({ conversations: [] }), { status: 200 });
}
throw new Error(`unexpected fetch: ${target}`);
};
const result = await validateProviderApiKey({
provider: "copilot-web",
apiKey: "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.test",
});
assert.equal(result.valid, true);
assert.equal(result.error, null);
});
test("copilot-web validator: cookie with access_token= is extracted", async () => {
let capturedAuth = "";
globalThis.fetch = async (url, init = {}) => {
if (String(url).includes("copilot.microsoft.com")) {
capturedAuth = ((init.headers as Record<string, string>) || {}).Authorization || "";
return new Response(JSON.stringify({}), { status: 200 });
}
throw new Error(`unexpected fetch: ${String(url)}`);
};
await validateProviderApiKey({
provider: "copilot-web",
apiKey: "access_token=eyJhbGciOiJIUzI1NiJ9.payload.sig; other_cookie=foo",
});
assert.equal(capturedAuth, "Bearer eyJhbGciOiJIUzI1NiJ9.payload.sig");
});
test("copilot-web validator: 401 → invalid token", async () => {
globalThis.fetch = async () => new Response("unauthorized", { status: 401 });
const result = await validateProviderApiKey({
provider: "copilot-web",
apiKey: "bad-token",
});
assert.equal(result.valid, false);
assert.match(result.error || "", /Invalid or expired access_token/i);
});
test("copilot-web validator: 500 → unavailable", async () => {
globalThis.fetch = async () => new Response("down", { status: 500 });
const result = await validateProviderApiKey({
provider: "copilot-web",
apiKey: "any-token",
});
assert.equal(result.valid, false);
assert.match(result.error || "", /Copilot unavailable \(500\)/i);
});
test("copilot-web validator: empty input → paste prompt", async () => {
globalThis.fetch = async () => {
throw new Error("should not fetch");
};
const result = await validateProviderApiKey({ provider: "copilot-web", apiKey: "" });
assert.equal(result.valid, false);
assert.match(result.error || "", /Paste your access_token/i);
});
// ─── t3-web validator ────────────────────────────────────────────────────────
test("t3-web validator: valid cookies → valid", async () => {
globalThis.fetch = async (url, init = {}) => {
if (String(url).includes("t3.chat")) {
return new Response("ok", { status: 200 });
}
throw new Error(`unexpected fetch: ${String(url)}`);
};
const result = await validateProviderApiKey({
provider: "t3-web",
apiKey: "cookies=__session=abc123; convexSessionId=def456",
});
assert.equal(result.valid, true);
assert.equal(result.error, null);
});
test("t3-web validator: 500 → unavailable", async () => {
globalThis.fetch = async () => new Response("down", { status: 500 });
const result = await validateProviderApiKey({
provider: "t3-web",
apiKey: "cookies=__session=abc",
});
assert.equal(result.valid, false);
assert.match(result.error || "", /t3\.chat unavailable \(500\)/i);
});
test("t3-web validator: valid cookies → passes through", async () => {
globalThis.fetch = async (url, init = {}) => {
if (String(url).includes("t3.chat")) {
return new Response("ok", { status: 200 });
}
throw new Error(`unexpected fetch: ${String(url)}`);
};
const result = await validateProviderApiKey({
provider: "t3-web",
apiKey: "__session=abc123; convex-session-id=def456",
});
assert.equal(result.valid, true);
assert.equal(result.error, null);
});
test("llama-cpp is classified as a self-hosted chat provider", async () => {
const { isSelfHostedChatProvider, isLocalProvider, providerAllowsOptionalApiKey } =
await import("../../src/shared/constants/providers.ts");
assert.equal(isSelfHostedChatProvider("llama-cpp"), true);
assert.equal(isLocalProvider("llama-cpp"), true);
assert.equal(providerAllowsOptionalApiKey("llama-cpp"), true);
});
// ─── Gitlawb Opengateway specialty validators ──────────────────────────────
test("gitlawb validator: accepts valid API key via chat/completions probe", async () => {
const calls: any[] = [];
globalThis.fetch = async (url, init = {}) => {
calls.push({ url: String(url), headers: init.headers || {}, body: init.body });
assert.equal(String(url), "https://opengateway.gitlawb.com/v1/xiaomi-mimo/chat/completions");
assert.equal((init.headers as Record<string, string>).Authorization, "Bearer glb-valid-key");
const body = JSON.parse(String(init.body));
assert.equal(body.model, "mimo-v2.5-pro");
assert.equal(body.messages[0].content, "test");
assert.equal(body.max_tokens, 1);
return new Response(JSON.stringify({ choices: [{ message: { content: "ok" } }] }), {
status: 200,
});
};
const result = await validateProviderApiKey({ provider: "gitlawb", apiKey: "glb-valid-key" });
assert.equal(result.valid, true);
assert.equal(calls.length, 1);
});
test("gitlawb validator: 400/422/429 treated as auth success", async () => {
for (const status of [400, 422, 429]) {
globalThis.fetch = async (url) => {
assert.equal(String(url), "https://opengateway.gitlawb.com/v1/xiaomi-mimo/chat/completions");
return new Response(JSON.stringify({ error: "bad request" }), { status });
};
const result = await validateProviderApiKey({ provider: "gitlawb", apiKey: "glb-key" });
assert.equal(result.valid, true, `status ${status} should pass auth`);
assert.equal(result.error, null, `status ${status} should not return error`);
}
});
test("gitlawb validator: rejects invalid API key (401)", async () => {
globalThis.fetch = async (url) => {
assert.equal(String(url), "https://opengateway.gitlawb.com/v1/xiaomi-mimo/chat/completions");
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
};
const result = await validateProviderApiKey({ provider: "gitlawb", apiKey: "glb-bad-key" });
assert.equal(result.valid, false);
assert.equal(result.error, "Invalid API key");
});
test("gitlawb validator: rejects invalid API key (403)", async () => {
globalThis.fetch = async (url) => {
assert.equal(String(url), "https://opengateway.gitlawb.com/v1/xiaomi-mimo/chat/completions");
return new Response(JSON.stringify({ error: "forbidden" }), { status: 403 });
};
const result = await validateProviderApiKey({ provider: "gitlawb", apiKey: "glb-bad-key" });
assert.equal(result.valid, false);
assert.equal(result.error, "Invalid API key");
});
test("gitlawb validator: surfaces network failures", async () => {
globalThis.fetch = async () => {
throw new Error("gitlawb opengateway offline");
};
const result = await validateProviderApiKey({ provider: "gitlawb", apiKey: "glb-key" });
assert.equal(result.valid, false);
assert.match(result.error || "", /gitlawb opengateway offline/i);
});
test("gitlawb validator: accepts custom baseUrl override", async () => {
globalThis.fetch = async (url, init = {}) => {
assert.equal(String(url), "https://custom-gateway.example.com/v1/xiaomi-mimo/chat/completions");
assert.equal((init.headers as Record<string, string>).Authorization, "Bearer glb-key");
return new Response(JSON.stringify({ choices: [{ message: { content: "ok" } }] }), {
status: 200,
});
};
const result = await validateProviderApiKey({
provider: "gitlawb",
apiKey: "glb-key",
providerSpecificData: {
baseUrl: "https://custom-gateway.example.com/v1/xiaomi-mimo",
},
});
assert.equal(result.valid, true);
});
// ─── Gitlawb-GMI (GMI Cloud) ─────────────────────────────────────────────
test("gitlawb-gmi validator: accepts valid API key via chat/completions probe", async () => {
const calls: any[] = [];
globalThis.fetch = async (url, init = {}) => {
calls.push({ url: String(url), headers: init.headers || {} });
assert.equal(String(url), "https://opengateway.gitlawb.com/v1/gmi-cloud/chat/completions");
assert.equal(
(init.headers as Record<string, string>).Authorization,
"Bearer glb-gmi-valid-key"
);
const body = JSON.parse(String(init.body));
assert.equal(body.model, "XiaomiMiMo/MiMo-V2.5-Pro");
assert.equal(body.messages[0].content, "test");
assert.equal(body.max_tokens, 1);
return new Response(JSON.stringify({ choices: [{ message: { content: "ok" } }] }), {
status: 200,
});
};
const result = await validateProviderApiKey({
provider: "gitlawb-gmi",
apiKey: "glb-gmi-valid-key",
});
assert.equal(result.valid, true);
assert.equal(calls.length, 1);
});
test("gitlawb-gmi validator: accepts 400/422/429 as auth success", async () => {
for (const status of [400, 422, 429]) {
globalThis.fetch = async (url) => {
assert.equal(String(url), "https://opengateway.gitlawb.com/v1/gmi-cloud/chat/completions");
return new Response(JSON.stringify({ error: "bad request" }), { status });
};
const result = await validateProviderApiKey({
provider: "gitlawb-gmi",
apiKey: "glb-gmi-key",
});
assert.equal(result.valid, true, `status ${status} should pass auth`);
}
});
test("gitlawb-gmi validator: rejects invalid API key (401)", async () => {
globalThis.fetch = async (url) => {
assert.equal(String(url), "https://opengateway.gitlawb.com/v1/gmi-cloud/chat/completions");
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
};
const result = await validateProviderApiKey({
provider: "gitlawb-gmi",
apiKey: "glb-gmi-bad-key",
});
assert.equal(result.valid, false);
assert.equal(result.error, "Invalid API key");
});
test("gitlawb-gmi validator: rejects invalid API key (403)", async () => {
globalThis.fetch = async (url) => {
assert.equal(String(url), "https://opengateway.gitlawb.com/v1/gmi-cloud/chat/completions");
return new Response(JSON.stringify({ error: "forbidden" }), { status: 403 });
};
const result = await validateProviderApiKey({
provider: "gitlawb-gmi",
apiKey: "glb-gmi-bad-key",
});
assert.equal(result.valid, false);
assert.equal(result.error, "Invalid API key");
});
test("gitlawb-gmi validator: surfaces network failures", async () => {
globalThis.fetch = async () => {
throw new Error("gitlawb-gmi opengateway offline");
};
const result = await validateProviderApiKey({
provider: "gitlawb-gmi",
apiKey: "glb-gmi-key",
});
assert.equal(result.valid, false);
assert.match(result.error || "", /gitlawb-gmi opengateway offline/i);
});
test("gitlawb-gmi validator: accepts custom baseUrl override", async () => {
globalThis.fetch = async (url, init = {}) => {
assert.equal(String(url), "https://custom-gateway.example.com/v1/gmi-cloud/chat/completions");
assert.equal((init.headers as Record<string, string>).Authorization, "Bearer glb-gmi-key");
return new Response(JSON.stringify({ choices: [{ message: { content: "ok" } }] }), {
status: 200,
});
};
const result = await validateProviderApiKey({
provider: "gitlawb-gmi",
apiKey: "glb-gmi-key",
providerSpecificData: {
baseUrl: "https://custom-gateway.example.com/v1/gmi-cloud",
},
});
assert.equal(result.valid, true);
});
// #3288 / #3758: qwen-web validation used to fall through to the generic
// OpenAI-compatible validator, which probed a non-existent `/api/v2/models` URL that
// answered with a 307 redirect — blocked by the outbound guard and mislabeled as an
// SSRF block. A specialty validator now probes the real session endpoint instead.
test("qwen-web validator probes /api/v2/user (not /api/v2/models) and returns valid on 200", async () => {
let probedUrl = "";
let sentHeaders: Record<string, string> = {};
globalThis.fetch = async (url, init = {}) => {
probedUrl = String(url);
sentHeaders = toPlainHeaders(init.headers);
// Qwen's /api/v2/user returns a real user object for a valid session. Since
// #3958 the validator inspects the body for `user` (Qwen answers 200 even for
// invalid tokens), so a valid response must carry one.
return new Response(JSON.stringify({ user: { id: "u-1", name: "Tester" } }), {
status: 200,
headers: { "content-type": "application/json" },
});
};
const result = await validateProviderApiKey({
provider: "qwen-web",
apiKey: "token=eyJqwen; cna=abc; ssxmod_itna=def",
});
assert.equal(probedUrl, "https://chat.qwen.ai/api/v2/user");
assert.ok(!probedUrl.includes("/api/v2/models"), "must not probe the bogus /api/v2/models URL");
assert.equal(sentHeaders.Authorization, "Bearer eyJqwen");
assert.equal(sentHeaders.source, "web");
assert.match(sentHeaders.Cookie, /token=eyJqwen/);
assert.equal(result.valid, true);
});
test("qwen-web validator reports an invalid session (401) without flagging a security block", async () => {
globalThis.fetch = async () =>
new Response(JSON.stringify({ error: "unauthorized" }), {
status: 401,
headers: { "content-type": "application/json" },
});
const result = await validateProviderApiKey({
provider: "qwen-web",
apiKey: "token=stale; cna=abc; ssxmod_itna=def",
});
assert.equal(result.valid, false);
assert.equal((result as { securityBlocked?: boolean }).securityBlocked ?? false, false);
assert.match(result.error ?? "", /invalid or expired/i);
});
test("qwen-web validator surfaces the WAF/anti-bot HTML challenge as a re-login hint", async () => {
globalThis.fetch = async () =>
new Response("<html>aliyun_waf</html>", {
status: 200,
headers: { "content-type": "text/html" },
});
const result = await validateProviderApiKey({
provider: "qwen-web",
apiKey: "token=eyJqwen; cna=abc; ssxmod_itna=def",
});
assert.equal(result.valid, false);
assert.match(result.error ?? "", /WAF|Cookie header/i);
});
// #3288 / #3758: a blocked redirect (REDIRECT_BLOCKED) to a PUBLIC host is benign — the
// redirect was never followed, so it must NOT be mislabeled as an SSRF security block.
// Only a redirect whose target is a private/internal host is a genuine security event.
test("isSecurityBlockError: public-host redirect block is NOT a security block", () => {
const publicRedirect = new SafeOutboundFetchError("Redirect blocked", {
code: "REDIRECT_BLOCKED",
url: "https://chat.qwen.ai/api/v2/models",
method: "GET",
attempts: 1,
status: 307,
location: "https://chat.qwen.ai/login",
isRetryable: false,
});
assert.equal(isSecurityBlockError(publicRedirect), false);
});
test("isSecurityBlockError: private-host redirect block IS a security block", () => {
const privateRedirect = new SafeOutboundFetchError("Redirect blocked", {
code: "REDIRECT_BLOCKED",
url: "https://api.example.com/probe",
method: "GET",
attempts: 1,
status: 302,
location: "http://169.254.169.254/latest/meta-data/",
isRetryable: false,
});
assert.equal(isSecurityBlockError(privateRedirect), true);
});
test("isSecurityBlockError: a URL-guard block remains a security block", () => {
const guardBlock = new SafeOutboundFetchError("Blocked private host", {
code: "URL_GUARD_BLOCKED",
url: "http://10.0.0.5/internal",
method: "GET",
attempts: 1,
isRetryable: false,
});
assert.equal(isSecurityBlockError(guardBlock), true);
});