mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-06 07:12:12 +03:00
* chore(release): open v3.8.37 development cycle * chore(ci): harden release flow — ratchet decoupling, fast-path drift gates, build-scope guard, heap default (#5054) Implements improvements 1-4 from the v3.8.36 release benchmark (_tasks/release-bench/v3.8.36/PLANO-MELHORIA.md): 1. Quality Ratchet decoupled from flaky coverage (ci.yml): the shard→coverage→ratchet chain meant a single flaky Coverage Shard SKIPPED the whole Quality Ratchet on the release PR (v3.8.36 #4854), so cycle drift only surfaced post-merge in #5029. The job now runs on !cancelled(); coverage download is continue-on-error and the ratchet runs --allow-missing, so the DETERMINISTIC gates (eslint/complexity/cognitive/duplication/ codeql) stay blocking even when coverage is unavailable. 2. Fast-path drift gates (quality.yml PR→release): added check:complexity, check:cognitive-complexity, and a new lightweight check:pack-policy (pack-artifact unexpected-files check WITHOUT a build, via --policy-only) so drift + stray-tarball-file regressions are caught/rebaselined PER-PR instead of cascading onto the release PR. 3. Build heap default 4096→8192 MB (build-next-isolated.mjs): the clean graph peaks ~3.9 GB and brushed the old 4 GB ceiling; 8 GB gives headroom. Comment notes heap is NOT the fix for a poisoned scope (run check:build-scope instead). 4. check:build-scope gate (new): fails if .ts/.tsx/.js/.jsx files in the tsconfig scope exceed a threshold — catches worktrees/cruft leaking into the build scope (the v3.8.36 OOM root cause: 355,215 vs 4,547 files) BEFORE it detonates next build. Wired into the fast-path. * fix(auth): only trust forwarding headers from loopback TCP peers (#4689) Integrated into release/v3.8.37 — loopback-gated forwarding headers (IP spoofing fix). Cherry-picked onto current release tip; ipUtils.test.ts 9/9 green. * fix(codex): treat OAuth 401 as unrecoverable refresh failure (#4686) Integrated into release/v3.8.37 — codex OAuth 401 treated as unrecoverable refresh. Cherry-picked onto release tip; token-refresh-service.test.ts 38/38 green. * fix(translator): preserve reasoning_effort for non-Copilot Responses clients (#4688) Integrated into release/v3.8.37 — preserve reasoning_effort for non-Copilot Responses clients. Cherry-picked onto release tip; tests 47/47 green. * fix(translator): coerce tool descriptions to strings in OpenAI normalization (#4675) Integrated into release/v3.8.37 — coerce tool descriptions to strings in OpenAI normalization. Cherry-picked onto release tip; tests 3/3 green. * feat(sse): x-omniroute-strip-reasoning header to drop reasoning_content (#4678) Integrated into release/v3.8.37 — x-omniroute-strip-reasoning header. Cherry-picked onto release tip (resolved chatCore.ts/headers.ts adjacency conflict, kept resolveCompressionHeader + isStripReasoningRequested); tests 8/8 green. * fix(combo): flatten Anthropic tool messages + tool history to prevent upstream 503 (#4648) Integrated into release/v3.8.37 — flattenToolHistory helper (combo anti-503). Cherry-picked onto release tip; tests 9/9 green. * feat(headroom): proxy lifecycle management + dashboard UI (Docker sidecar supported) (#4649) Integrated into release/v3.8.37 — headroom proxy lifecycle (status/start/stop, local-only + spawn-capable per Rules #15/#17). Cherry-picked onto release tip; lifecycle 7/7 + route-guard 43/43 + check:cycles green. * feat(cli): multi-model support for Factory Droid CLI (#4682) Integrated into release/v3.8.37 — Factory Droid multi-model support. Cherry-picked onto release tip (kept readJsoncConfig + droidCustomModels imports); droid-custom-models 11/11 green. * fix(providers): require Default Model in compatible-provider API-key setup (#4641) Integrated into release/v3.8.37 — require Default Model in compatible-provider API-key setup. Cherry-picked fix + test-move onto release tip (kept release providerSpecificData + QuotaScrapingFields; fixed moved-test import path; baseline rebaseline unneeded, 865<866); UI test 2/2 green. * fix(dashboard): stop double-masking already-masked API key in list (E2E 3/9 regression) (#4671) Integrated into release/v3.8.37 — render server-masked key verbatim (drop redundant maskKey call). Note: release's maskKey already guards '****' (since v3.8.34), so this is a safe simplification; added a contract test pinning the **** passthrough invariant (2/2 green, would fail against the pre-guard maskKey = the historical double-mask bug). * chore(quality): rebaseline file-size for rc17 PR batch drift Own growth from the merged rc17 PRs (#4678/#4686/#4688) at existing chokepoints — cohesive, not extractable: - open-sse/handlers/responseSanitizer.ts 1103->1122 (SanitizeOpenAIResponseOptions + stripReasoning, #4678) - open-sse/services/tokenRefresh.ts 2070->2090 (codex 401 unrecoverable-refresh guard, #4686) - tests/unit/token-refresh-service.test.ts 1322->1353 (401 regression case, #4686) - tests/unit/translator-openai-responses-req.test.ts 1047->1050 (reasoning_effort assertion, #4688) * docs(env): document HEADROOM_URL in .env.example + ENVIRONMENT.md The headroom proxy lifecycle (#4649) reads HEADROOM_URL (src/lib/headroom/detect.ts, default http://localhost:8787) but it was missing from the env contract, tripping check:env-doc-sync. Adds the var to both .env.example (commented, has a default) and the Proxy Health table in ENVIRONMENT.md. * fix(sse): stream writer mock abort() returns a Promise (#4788) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; tests green. * fix(cli): fall back to default data dir when DATA_DIR is not writable (#4767) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; tests green. * fix(oauth): verify Cursor installation on Linux before auto-import (#4770) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; tests green. * fix(sse): track Ollama streaming usage from raw NDJSON chunks (#4754) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; tests green. * fix(sse): strip enumDescriptions from antigravity tool schema (#4740) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; tests green. * fix(sse): include low-level cause details in formatProviderError (#4741) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; tests green. * fix(translator): strip x-anthropic-billing-header in claude-to-openai (#4728) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; tests green. * fix(sse): gate Kiro image attachments behind a Claude-capability check (#4763) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; tests green. * fix(sse): read Antigravity usage from the response.usageMetadata envelope (#4785) Integrated into release/v3.8.37 — Antigravity response.usageMetadata envelope. Cherry-picked onto release tip (resolved test-tail adjacency with #4754 Ollama block); usage-extractor 23/23 green. * fix(api): fall back to existing access token for any OAuth provider on refresh failure (#4786) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; tests green. * fix(cli): verify launchd registration + skip self-SIGTERM in macOS autostart (#4765) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; tests green. * fix(executors): anthropic-compatible-* gateways get Bearer alongside x-api-key (#4729) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; tests green. * fix(sse): json_schema fallback for OpenAI-compatible providers (#4766) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; tests green. * fix(sse): use workos auth token shape for cline (#4787) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; tests green. * feat(sse): parse Gemini CLI 429 retryDelay from structured RetryInfo (#4738) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; tests green. * fix(sse): finalize tool_calls finish_reason on early stream end in OpenAI Responses translator (#4764) Integrated into release/v3.8.37 — computeFinishReason finalizes tool_calls on early stream end (Responses translator). Cherry-picked onto release tip; responses-translation-fixes 29/29 green. * test(sse): golden-lock provider.ts translate-path across all providers (#4734) Integrated into release/v3.8.37 — golden-lock for provider.ts translate-path. Cherry-picked onto release tip; snapshot regenerated against the current provider set (UPDATE_GOLDEN=1, 167 entries); golden test 3/3 deterministic. * chore(quality): rebaseline file-size for rc17 leva2 PR batch drift Own growth from the merged leva2 PRs (cohesive, not extractable): - src/lib/usage/providerLimits.ts 950->955 (#4786) - open-sse/executors/default.ts NEW frozen @828 (#4729 + #4766 + #4787 header branches) - open-sse/translator/request/openai-to-kiro.ts 807->814 (#4763) - open-sse/translator/response/openai-responses.ts 923->937 (#4764) - tests/unit/executor-default-base.test.ts 1339->1440 (#4766) - tests/unit/translator-openai-to-kiro.test.ts 918->980 (#4763) * fix(dashboard): align Engine Combos editor engines with API schema (#4955) (#5062) The named-combos pipeline dropdown offered four engines (headroom, session-dedup, ccr, llmlingua) that stackedPipelineStepSchema rejects, so selecting one made PUT /api/context/combos/[id] return HTTP 400 while saveCombo swallowed the non-OK response (if (!res.ok) return). Editing the default 'Standard Savings' combo and changing an engine reproduced the 400. - Add canonical STACKED_PIPELINE_ENGINE_INTENSITIES next to the schema as the single source of truth; the client dropdown imports it so it can never drift from the discriminated union the API validates against. - Surface save errors and empty-name/empty-pipeline validation in the editor instead of failing silently. - Add a parity unit test asserting the UI engine map equals the schema union and that every (engine, intensity) the UI emits is accepted. * fix(sse): filter nameless hosted tools when converting Responses API to Chat format (#4789) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * fix(dashboard): keep desktop sidebar visible via explicit CSS class (#4812) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * fix(sse): strip enumDescriptions from Antigravity tool schemas (#4813) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * fix(dashboard): resolve passthrough model aliases by providerId in ModelSelectModal (#4815) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * fix(oauth): allow per-connection refresh lead-time override via providerSpecificData.refreshLeadMs (#4818) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * fix(sse): strip X-Stainless-* headers and normalize SDK User-Agent for OpenAI-compatible endpoints (#4820) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * fix(sse): strip Gemini built-in tools when functionDeclarations present in Antigravity envelope (#4821) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * fix(api): surface a Docker-localhost hint on provider-node validation connection errors (#4822) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * fix(sse): resolve bare model names to connection defaultModel before upstream calls (#4825) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * fix(build): trace-include sql.js sql-wasm.wasm in standalone bundle (#4839) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * fix(sse): strip Composer <|final|> sentinel markers leaking after Composer reasoning (#4842) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * fix(config): sync full SiliconFlow model list into registry (#4844) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * fix(sse): close reasoning before message content in Responses stream (#4848) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * fix(sse): reject unsupported Kiro [1m] context suffix (#4816) Integrated into release/v3.8.37 — cherry-picked onto release tip; test-tail conflict with #4763 resolved (kept both image + [1m] test blocks); CHANGELOG re-merged; 29/29 green. * fix(db): validate HuggingFace tokens via whoami-v2 auth probe (#4819) Integrated into release/v3.8.37 — defining commit re-homed onto the god-file-split validation module (validateHuggingFaceProvider in validation/openaiFormat.ts + map wiring); 115/115 green. * fix(sse): make anthropic-version default-guard case-insensitive (#4823) Integrated into release/v3.8.37 — conflict with #4729 Bearer-fallback resolved (kept both Bearer fallback + case-insensitive anthropic-version guard); 48/48 green. * fix(sse): sanitize Kiro tool schemas to avoid 400 "Improperly formed request" (#4847) Integrated into release/v3.8.37 — conflict in kiro-to-openai.ts resolved (kept release fallbackToolCallId + adopted #1375 toolNameMap remap); 7/7 green. * feat(sse): add GPT-4 to the GitHub Copilot provider (#4798) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * feat(sse): add GPT-4o mini to GitHub Copilot provider (#4797) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * feat(api): add MiniMax-M3 pricing row (#4814) Integrated into release/v3.8.37 — pricing row re-homed onto god-file-split pricing/regional.ts (pricing.ts is now a barrel); 4/4 green. * fix(cli): save runtime deps with --save-exact so a sibling install can't prune them (#4841) Integrated into release/v3.8.37 — trayRuntime conflict resolved (kept release SYSTRAY_SPEC + added --save-exact); 2/2 green. * fix(sse): preserve required fields in antigravity tool schemas (#4843) Integrated into release/v3.8.37 — conflict resolved (kept #4740/#4813 enumDescriptions strip + typed normalizeSchemaTypes, added required-preservation helpers; test-tail merged keeping both enumDescriptions + required tests); 7/7 green. * chore(quality): rebaseline file-size for rc17b leva3 PR batch drift * fix(sse): strip reasoning blobs from agentic context to prevent O(n^2) token growth (#4849) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * fix(sse): unwrap Qoder HTTP 200 SSE error envelope so fallback can trigger (#4850) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * fix(sse): strip temperature for Claude models with extended thinking (#4853) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * fix(sse): emit valid concatenable kiro tool_calls.arguments deltas (#4855) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * feat(sse): add toggleable tool-source diagnostics (#4856) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * fix(sse): redact api key from the AUTH debug log in the chat handler (#4858) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * fix(sse): forward AI SDK image parts in Responses translator (#4859) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * fix(sse): resolve custom combos by id and case-insensitive name (#4446) (#4869) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * fix(sse): exclude WS bridge controller-closed error from provider breaker (#4602) (#4870) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * feat(providers): add xAI Grok inbound translators and thinking patcher (#4910) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * feat(embeddings): add dimensions override field to embedding combos (#4913) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * feat(oauth): Codex bulk-import endpoint — POST /api/oauth/codex/import (#4914) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * fix(antigravity): retry transient upstream failures (#4941) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * fix(sse): surface malformed HTTP-200 upstream responses (#4942) Integrated into release/v3.8.37 — cherry-picked defining commit onto release tip; CHANGELOG re-merged; tests green. * fix(sse): normalize Codex custom tools (apply_patch) to { input: string } schema (#4862) Integrated into release/v3.8.37 — conflict in request/openai-responses.ts resolved (kept #4789 nameless-tool skip + added #1007 custom-tool {input:string} normalization); 48/48 green incl. #4789/#4859 regression. * fix(sse): dense, deterministic output ordering in Responses API response.completed (#4906) Integrated into release/v3.8.37 — manual integration with #4862 in response/openai-responses.ts (custom-tool funcItem + dense recordCompletedItem). Fixed a latent #4848 interaction: the close-reasoning-before-message guard force-closed <think>-tag reasoning prematurely, which dense output (#4906) then snapshotted as a partial buffer ("plan" vs "planning") — scoped the guard to native reasoning_content (!inThinking) in BOTH transformer + translator paths. Full Responses suite 203/203 green incl. #4848/#4862 regression. * feat(sse): auto-promote successful combo model to position #1 (#4852) Integrated into release/v3.8.37 — dropped the stale file-size-baseline.json hunk (re-derived against the rc17b rebaseline); code+test applied clean; 13/13 green. * feat(providers): add Pioneer AI (Fastino Labs) provider (#4909) Integrated into release/v3.8.37 — providers.ts apikey block re-homed onto god-file-split src/shared/constants/providers/apikey/frontier-labs.ts (inline APIKEY_PROVIDERS no longer exists); registry/pioneer + providers/index.ts applied clean; 6/6 green. * add DGrid AI gateway provider (#4931) Integrated into release/v3.8.37 — rebased the contributor's commit onto the release tip; providers.ts god-file-split conflict resolved by relocating the dgrid APIKEY_PROVIDERS entry into apikey/gateways.ts; CHANGELOG added. 7/7 green. Thanks @dgridOP! * chore(quality): rebaseline file-size for rc17b leva4 PR batch drift * docs(routing): sync combo strategy docs for Fusion (17 strategies) (#5067) Fusion (16th strategy, panel fan-out + judge synthesis) and headroom shipped but the strategy-count docs were stale (14/15) and omitted both. Update every combo-strategy reference to the canonical 17, add fusion + headroom to all strategy tables, and add a dedicated Fusion section to AUTO-COMBO.md documenting judgeModel / fusionTuning config + an example. - CLAUDE.md, README.md, FEATURES.md, RESILIENCE_GUIDE.md, ARCHITECTURE.md, OPEN_SSE_ARCHITECTURE.md, OMNIROUTE_VS_ALTERNATIVES.md, docs/README.md, request-pipeline.mmd: 14/15 -> 17, list fusion + headroom - docs/routing/AUTO-COMBO.md: strategy table + new Fusion strategy section - docs/openapi.yaml: add reset-window, headroom, fusion to the strategy enum * fix(oauth): classify /api/oauth/cursor/auto-import as local-only (route-guard) (#5070) The Cursor auto-import route runs execFile("which", ["cursor"]) to verify a local Cursor install before importing credentials — a child-process spawn. The check:route-guard-membership gate (Hard Rules #15/#17) flagged it as an unclassified spawn-capable route: reachable past the loopback gate, an RCE-via-tunnel surface (a leaked JWT over a tunnel could trigger the spawn). Classify the specific path in LOCAL_ONLY_API_PREFIXES so loopback enforcement runs unconditionally before any auth check. Scoped to the exact path — the rest of /api/oauth/ (browser redirect/callback flows) stays remote-reachable. TDD: added a failing-then-passing assertion in route-guard-local-prefix.test.ts (classification + an over-broadening guard proving sibling OAuth paths stay remote). check:route-guard-membership now reports 0 new gaps. * chore(release): v3.8.37 — 2026-06-26 --------- Co-authored-by: dgridOP <dgrid_op@outlook.com>
2801 lines
102 KiB
TypeScript
2801 lines
102 KiB
TypeScript
import test from "node:test";
|
|
import assert from "node:assert/strict";
|
|
|
|
const {
|
|
validateProviderApiKey,
|
|
validateClaudeCodeCompatibleProvider,
|
|
validateCommandCodeProvider,
|
|
isSecurityBlockError,
|
|
} = await import("../../src/lib/providers/validation.ts");
|
|
|
|
const { SafeOutboundFetchError } = await import("../../src/shared/network/safeOutboundFetch.ts");
|
|
|
|
const { __setTlsFetchOverrideForTesting: __setPplxTlsFetchOverride } =
|
|
await import("../../open-sse/services/perplexityTlsClient.ts");
|
|
|
|
const { __setTlsFetchOverrideForTesting: __setGrokTlsFetchOverride } =
|
|
await import("../../open-sse/services/grokTlsClient.ts");
|
|
|
|
const { COMMAND_CODE_VERSION } = await import("../../open-sse/executors/commandCode.ts");
|
|
|
|
const originalFetch = globalThis.fetch;
|
|
|
|
test.afterEach(() => {
|
|
globalThis.fetch = originalFetch;
|
|
__setPplxTlsFetchOverride(null);
|
|
__setGrokTlsFetchOverride(null);
|
|
});
|
|
|
|
function toPlainHeaders(headers: any) {
|
|
if (headers instanceof Headers) return Object.fromEntries(headers.entries());
|
|
return Object.fromEntries(
|
|
Object.entries(headers || {}).map(([key, value]) => [key, String(value)])
|
|
);
|
|
}
|
|
|
|
function metaAiSseText(content: string, streamingState = "DONE") {
|
|
return `event: next
|
|
data: ${JSON.stringify({
|
|
data: {
|
|
sendMessageStream: {
|
|
__typename: "AssistantMessage",
|
|
id: "meta-msg-1",
|
|
content,
|
|
streamingState,
|
|
error:
|
|
streamingState === "ERROR"
|
|
? { message: content, code: null, stack: "Error: " + content }
|
|
: null,
|
|
contentRenderer: { __typename: "TextContentRenderer", text: content },
|
|
},
|
|
},
|
|
})}
|
|
|
|
event: complete
|
|
data:
|
|
|
|
`;
|
|
}
|
|
|
|
test("specialty provider validators cover Deepgram, AssemblyAI, ElevenLabs and Inworld branches", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
const headers = init.headers || {};
|
|
|
|
if (target.match(/deepgram/i)) {
|
|
assert.equal(headers.Authorization, "Token dg-key");
|
|
return new Response(JSON.stringify({ ok: true }), { status: 200 });
|
|
}
|
|
if (target.match(/assemblyai/i)) {
|
|
assert.equal(headers.Authorization, "aa-key");
|
|
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 403 });
|
|
}
|
|
if (target.match(/elevenlabs/i)) {
|
|
return new Response(JSON.stringify({ voices: [] }), { status: 200 });
|
|
}
|
|
if (target.match(/inworld/i)) {
|
|
return new Response(JSON.stringify({ error: "bad request" }), { status: 400 });
|
|
}
|
|
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const deepgram = await validateProviderApiKey({ provider: "deepgram", apiKey: "dg-key" });
|
|
const assembly = await validateProviderApiKey({ provider: "assemblyai", apiKey: "aa-key" });
|
|
const eleven = await validateProviderApiKey({ provider: "elevenlabs", apiKey: "el-key" });
|
|
const inworld = await validateProviderApiKey({ provider: "inworld", apiKey: "iw-key" });
|
|
|
|
assert.equal(deepgram.valid, true);
|
|
assert.equal(assembly.error, "Invalid API key");
|
|
assert.equal(eleven.valid, true);
|
|
assert.equal(inworld.valid, true);
|
|
});
|
|
|
|
test("validateCommandCodeProvider ignores caller baseUrl and chatPath overrides", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
assert.equal(String(url), "https://api.commandcode.ai/alpha/generate");
|
|
const headers = init.headers as Record<string, string>;
|
|
assert.equal(headers.Authorization, "Bearer cc-key");
|
|
const body = JSON.parse(String(init.body));
|
|
assert.equal(body.params.model, "command-code-validation-model");
|
|
return new Response(JSON.stringify({ ok: true }), { status: 200 });
|
|
};
|
|
|
|
const result = await validateCommandCodeProvider({
|
|
apiKey: "cc-key",
|
|
providerSpecificData: {
|
|
baseUrl: "https://evil.example/api",
|
|
chatPath: "/v1/chat/completions",
|
|
validationModelId: "command-code-validation-model",
|
|
},
|
|
});
|
|
|
|
assert.equal(result.valid, true);
|
|
});
|
|
|
|
test("validateCommandCodeProvider defaults probe model to DeepSeek flash", async () => {
|
|
globalThis.fetch = async (_url, init = {}) => {
|
|
const body = JSON.parse(String(init.body));
|
|
assert.equal(body.params.model, "deepseek/deepseek-v4-flash");
|
|
return new Response("", { status: 400 });
|
|
};
|
|
|
|
const result = await validateCommandCodeProvider({ apiKey: "cc-key" });
|
|
|
|
assert.deepEqual(result, { valid: true, error: null });
|
|
});
|
|
|
|
test("specialty providers surface network failures and non-auth upstream failures", async () => {
|
|
globalThis.fetch = async (url) => {
|
|
const target = String(url);
|
|
if (target.match(/deepgram/i)) {
|
|
throw new Error("deepgram offline");
|
|
}
|
|
if (target.match(/elevenlabs/i)) {
|
|
return new Response(JSON.stringify({ error: "server" }), { status: 500 });
|
|
}
|
|
if (target.match(/inworld/i)) {
|
|
return new Response(JSON.stringify({ error: "forbidden" }), { status: 403 });
|
|
}
|
|
if (target.match(/longcat/i)) {
|
|
throw new Error("longcat offline");
|
|
}
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const deepgram = await validateProviderApiKey({ provider: "deepgram", apiKey: "dg-key" });
|
|
const eleven = await validateProviderApiKey({ provider: "elevenlabs", apiKey: "el-key" });
|
|
const inworld = await validateProviderApiKey({ provider: "inworld", apiKey: "iw-key" });
|
|
const longcat = await validateProviderApiKey({ provider: "longcat", apiKey: "lc-key" });
|
|
|
|
assert.equal(deepgram.error, "deepgram offline");
|
|
assert.equal(eleven.error, "Validation failed: 500");
|
|
assert.equal(inworld.error, "Invalid API key");
|
|
assert.equal(longcat.error, "longcat offline");
|
|
});
|
|
|
|
test("embedding and rerank specialty validators cover Voyage AI and Jina AI", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
|
|
if (target === "https://api.voyageai.com/v1/embeddings") {
|
|
assert.equal((init.headers as Record<string, string>).Authorization, "Bearer voyage-key");
|
|
const body = JSON.parse(String(init.body));
|
|
assert.equal(body.model, "voyage-4-large");
|
|
return new Response(JSON.stringify({ data: [{ embedding: [0.1, 0.2] }] }), { status: 200 });
|
|
}
|
|
|
|
if (target === "https://api.jina.ai/v1/rerank") {
|
|
assert.equal((init.headers as Record<string, string>).Authorization, "Bearer jina-key");
|
|
const body = JSON.parse(String(init.body));
|
|
assert.equal(body.model, "jina-reranker-v3");
|
|
return new Response(JSON.stringify({ results: [{ index: 0, relevance_score: 0.99 }] }), {
|
|
status: 200,
|
|
});
|
|
}
|
|
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const voyage = await validateProviderApiKey({ provider: "voyage-ai", apiKey: "voyage-key" });
|
|
const jina = await validateProviderApiKey({ provider: "jina-ai", apiKey: "jina-key" });
|
|
|
|
assert.equal(voyage.valid, true);
|
|
assert.equal(jina.valid, true);
|
|
});
|
|
|
|
test("AWS Polly specialty validator signs DescribeVoices with SigV4", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
const headers = init.headers as Record<string, string>;
|
|
|
|
assert.equal(target, "https://polly.us-east-2.amazonaws.com/v1/voices?Engine=standard");
|
|
assert.match(
|
|
headers.Authorization,
|
|
/^AWS4-HMAC-SHA256 Credential=AKIA_POLLY\/\d{8}\/us-east-2\/polly\/aws4_request,/
|
|
);
|
|
assert.equal(headers.host, "polly.us-east-2.amazonaws.com");
|
|
assert.equal(headers["x-amz-content-sha256"].length, 64);
|
|
return new Response(JSON.stringify({ Voices: [] }), { status: 200 });
|
|
};
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "aws-polly",
|
|
apiKey: "aws-secret",
|
|
providerSpecificData: {
|
|
accessKeyId: "AKIA_POLLY",
|
|
region: "us-east-2",
|
|
},
|
|
});
|
|
|
|
assert.equal(result.valid, true);
|
|
});
|
|
|
|
test("AWS Polly specialty validator requires an access key id", async () => {
|
|
const result = await validateProviderApiKey({
|
|
provider: "aws-polly",
|
|
apiKey: "aws-secret",
|
|
providerSpecificData: {
|
|
region: "us-east-2",
|
|
},
|
|
});
|
|
|
|
assert.equal(result.error, "Missing AWS accessKeyId");
|
|
});
|
|
|
|
test("embedding and rerank specialty validators surface auth failures for Voyage AI and Jina AI", async () => {
|
|
globalThis.fetch = async (url) => {
|
|
const target = String(url);
|
|
if (target === "https://api.voyageai.com/v1/embeddings") {
|
|
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
|
|
}
|
|
if (target === "https://api.jina.ai/v1/rerank") {
|
|
return new Response(JSON.stringify({ error: "forbidden" }), { status: 403 });
|
|
}
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const voyage = await validateProviderApiKey({ provider: "voyage-ai", apiKey: "voyage-key" });
|
|
const jina = await validateProviderApiKey({ provider: "jina-ai", apiKey: "jina-key" });
|
|
|
|
assert.equal(voyage.error, "Invalid API key");
|
|
assert.equal(jina.error, "Invalid API key");
|
|
});
|
|
|
|
test("gitlab specialty validator accepts PAT auth on the direct access endpoint", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
assert.equal(String(url), "https://gitlab.com/api/v4/code_suggestions/direct_access");
|
|
assert.equal((init.headers as Record<string, string>).Authorization, "Bearer glpat-test");
|
|
return new Response(JSON.stringify({ token: "short-lived" }), { status: 200 });
|
|
};
|
|
|
|
const result = await validateProviderApiKey({ provider: "gitlab", apiKey: "glpat-test" });
|
|
assert.equal(result.valid, true);
|
|
});
|
|
|
|
test("gitlab specialty validator treats 401 as invalid PAT", async () => {
|
|
globalThis.fetch = async () =>
|
|
new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
|
|
|
|
const result = await validateProviderApiKey({ provider: "gitlab", apiKey: "glpat-bad" });
|
|
assert.equal(result.error, "Invalid API key");
|
|
});
|
|
|
|
test("web-cookie provider validators accept valid Grok, Perplexity, Blackbox and Muse Spark session cookies", async () => {
|
|
const calls = [];
|
|
|
|
// Grok now uses tlsFetchGrok (TLS-impersonating client) to bypass Cloudflare Enterprise.
|
|
let grokTlsCall: { url: string; options: Record<string, unknown> } | null = null;
|
|
__setGrokTlsFetchOverride(async (url, options) => {
|
|
grokTlsCall = { url, options };
|
|
return { status: 200, headers: new Headers(), text: null, body: null };
|
|
});
|
|
|
|
// Perplexity now uses tlsFetchPerplexity (TLS-impersonating client) instead of globalThis.fetch
|
|
// to bypass Cloudflare Enterprise. Use the test-only override hook to intercept calls.
|
|
let pplxTlsCall: { url: string; options: Record<string, unknown> } | null = null;
|
|
__setPplxTlsFetchOverride(async (url, options) => {
|
|
pplxTlsCall = { url, options };
|
|
return { status: 200, headers: new Headers(), text: null, body: null };
|
|
});
|
|
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
calls.push({ url: target, init });
|
|
|
|
if (target.includes("app.blackbox.ai/api/auth/session")) {
|
|
return new Response(
|
|
JSON.stringify({
|
|
user: { id: "bb-user-1", email: "premium@example.com" },
|
|
}),
|
|
{ status: 200 }
|
|
);
|
|
}
|
|
if (target.includes("app.blackbox.ai/api/check-subscription")) {
|
|
return new Response(
|
|
JSON.stringify({
|
|
hasActiveSubscription: true,
|
|
isTrialSubscription: false,
|
|
plan: "pro",
|
|
}),
|
|
{ status: 200 }
|
|
);
|
|
}
|
|
if (target.includes("meta.ai/api/graphql")) {
|
|
return new Response(metaAiSseText("Muse Spark says hello"), {
|
|
status: 200,
|
|
headers: { "Content-Type": "text/event-stream" },
|
|
});
|
|
}
|
|
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const grok = await validateProviderApiKey({ provider: "grok-web", apiKey: "sso=grok-cookie" });
|
|
const perplexity = await validateProviderApiKey({
|
|
provider: "perplexity-web",
|
|
apiKey: "__Secure-next-auth.session-token=pplx-cookie",
|
|
});
|
|
const blackbox = await validateProviderApiKey({
|
|
provider: "blackbox-web",
|
|
apiKey: "__Secure-authjs.session-token=bb-cookie",
|
|
});
|
|
const museSpark = await validateProviderApiKey({
|
|
provider: "muse-spark-web",
|
|
apiKey: "abra_sess=meta-cookie",
|
|
});
|
|
|
|
assert.equal(grok.valid, true);
|
|
assert.equal(perplexity.valid, true);
|
|
assert.equal(blackbox.valid, true);
|
|
assert.equal(museSpark.valid, true);
|
|
|
|
const blackboxSessionCall = calls.find((call) =>
|
|
call.url.includes("app.blackbox.ai/api/auth/session")
|
|
);
|
|
const blackboxSubscriptionCall = calls.find((call) =>
|
|
call.url.includes("app.blackbox.ai/api/check-subscription")
|
|
);
|
|
const museSparkCall = calls.find((call) => call.url.includes("meta.ai/api/graphql"));
|
|
|
|
// Grok goes through tlsFetchGrok (TLS override), not globalThis.fetch.
|
|
assert.ok(grokTlsCall, "grok TLS override was called");
|
|
assert.ok(grokTlsCall!.url.includes("grok.com/rest/app-chat/conversations/new"));
|
|
assert.equal(
|
|
(grokTlsCall!.options.headers as Record<string, string>)["Cookie"],
|
|
"sso=grok-cookie"
|
|
);
|
|
const grokBody = JSON.parse(String(grokTlsCall!.options.body || "{}"));
|
|
assert.equal(grokBody.modeId, "fast");
|
|
assert.equal("modelName" in grokBody, false);
|
|
assert.equal("modelMode" in grokBody, false);
|
|
// Perplexity goes through tlsFetchPerplexity (TLS override), not globalThis.fetch.
|
|
// options.headers is a plain object; the validator sets Cookie from the session token.
|
|
assert.ok(pplxTlsCall, "perplexity TLS override was called");
|
|
assert.ok(pplxTlsCall!.url.includes("perplexity.ai/rest/sse/perplexity_ask"));
|
|
assert.equal(
|
|
(pplxTlsCall!.options.headers as Record<string, string>)["Cookie"],
|
|
"__Secure-next-auth.session-token=pplx-cookie"
|
|
);
|
|
assert.equal(blackboxSessionCall?.init.headers.Cookie, "__Secure-authjs.session-token=bb-cookie");
|
|
assert.equal(
|
|
blackboxSubscriptionCall?.init.headers.Cookie,
|
|
"__Secure-authjs.session-token=bb-cookie"
|
|
);
|
|
assert.equal(museSparkCall?.init.headers.Cookie, "abra_sess=meta-cookie");
|
|
assert.equal(museSparkCall?.init.headers["X-FB-Friendly-Name"], "useEctoSendMessageSubscription");
|
|
});
|
|
|
|
test("web-cookie provider validators surface auth and subscription failures", async () => {
|
|
// Perplexity uses tlsFetchPerplexity (TLS-impersonating client). Return 403 to simulate
|
|
// an invalid session cookie so the validator emits the expected error message.
|
|
__setPplxTlsFetchOverride(async () => {
|
|
return { status: 403, headers: new Headers(), text: null, body: null };
|
|
});
|
|
__setGrokTlsFetchOverride(async () => {
|
|
return { status: 401, headers: new Headers(), text: "Unauthorized", body: null };
|
|
});
|
|
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
if (target.includes("app.blackbox.ai/api/auth/session")) {
|
|
const cookie = (init.headers as Record<string, string>)?.Cookie || "";
|
|
if (cookie.includes("expired-cookie")) {
|
|
return new Response("null", { status: 200 });
|
|
}
|
|
return new Response(
|
|
JSON.stringify({
|
|
user: { id: "bb-user-2", email: "free@example.com" },
|
|
}),
|
|
{ status: 200 }
|
|
);
|
|
}
|
|
if (target.includes("app.blackbox.ai/api/check-subscription")) {
|
|
return new Response(
|
|
JSON.stringify({
|
|
hasActiveSubscription: false,
|
|
isTrialSubscription: false,
|
|
previouslySubscribed: true,
|
|
plan: "free",
|
|
}),
|
|
{ status: 200 }
|
|
);
|
|
}
|
|
if (target.includes("meta.ai/api/graphql")) {
|
|
return new Response(metaAiSseText("Authentication required to send messages", "ERROR"), {
|
|
status: 200,
|
|
headers: { "Content-Type": "text/event-stream" },
|
|
});
|
|
}
|
|
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const grok = await validateProviderApiKey({ provider: "grok-web", apiKey: "grok-cookie" });
|
|
const perplexity = await validateProviderApiKey({
|
|
provider: "perplexity-web",
|
|
apiKey: "pplx-cookie",
|
|
});
|
|
const blackboxExpired = await validateProviderApiKey({
|
|
provider: "blackbox-web",
|
|
apiKey: "expired-cookie",
|
|
});
|
|
const blackboxNoSubscription = await validateProviderApiKey({
|
|
provider: "blackbox-web",
|
|
apiKey: "free-account-cookie",
|
|
});
|
|
const museSpark = await validateProviderApiKey({
|
|
provider: "muse-spark-web",
|
|
apiKey: "meta-cookie",
|
|
});
|
|
|
|
assert.match(grok.error || "", /Invalid SSO cookie/i);
|
|
assert.match(perplexity.error || "", /Invalid Perplexity session cookie/i);
|
|
assert.match(blackboxExpired.error || "", /Invalid Blackbox session cookie/i);
|
|
assert.match(blackboxNoSubscription.error || "", /no active paid subscription/i);
|
|
assert.match(museSpark.error || "", /Invalid Meta AI session cookie/i);
|
|
});
|
|
|
|
test("grok-web validator: full DevTools cookie blob is parsed for the sso value", async () => {
|
|
let capturedCookie = "";
|
|
__setGrokTlsFetchOverride(async (_url, options) => {
|
|
capturedCookie = ((options.headers as Record<string, string>) || {}).Cookie || "";
|
|
return { status: 200, headers: new Headers(), text: null, body: null };
|
|
});
|
|
|
|
const blob = "i18nextLng=en; stblid=foo; __cf_bm=bar; sso=eyJTARGET.abc.def; cf_clearance=baz;";
|
|
const result = await validateProviderApiKey({ provider: "grok-web", apiKey: blob });
|
|
|
|
assert.equal(result.valid, true);
|
|
assert.equal(capturedCookie, "sso=eyJTARGET.abc.def");
|
|
});
|
|
|
|
test("grok-web validator: empty/missing sso in input returns 'Missing sso cookie'", async () => {
|
|
__setGrokTlsFetchOverride(async () => {
|
|
throw new Error("validator should short-circuit before fetching");
|
|
});
|
|
const result = await validateProviderApiKey({
|
|
provider: "grok-web",
|
|
apiKey: "foo=1; bar=2;",
|
|
});
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /Missing sso cookie/i);
|
|
});
|
|
|
|
test("grok-web validator: non-auth 403 is reported as failure with upstream body, not silently passed", async () => {
|
|
__setGrokTlsFetchOverride(async () => {
|
|
return {
|
|
status: 403,
|
|
headers: new Headers(),
|
|
text: JSON.stringify({ error: { code: 7, message: "Model is not found", details: [] } }),
|
|
body: null,
|
|
};
|
|
});
|
|
|
|
const result = await validateProviderApiKey({ provider: "grok-web", apiKey: "good-cookie" });
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /Grok rejected validation \(403\)/);
|
|
assert.match(result.error || "", /Model is not found/);
|
|
});
|
|
|
|
test("grok-web validator: generic non-auth 403 maps to IP-reputation guidance, not 'invalid cookie' (#3474)", async () => {
|
|
__setGrokTlsFetchOverride(async () => {
|
|
return { status: 403, headers: new Headers(), text: "Forbidden", body: null };
|
|
});
|
|
|
|
const result = await validateProviderApiKey({ provider: "grok-web", apiKey: "any-cookie" });
|
|
assert.equal(result.valid, false);
|
|
// A bare/non-auth 403 from grok.com is almost always an anti-bot/IP-reputation
|
|
// block — the cookie itself is likely fine. The message must point the user to a
|
|
// residential IP or proxy, NOT tell them the cookie is invalid.
|
|
assert.match(result.error || "", /residential IP|proxy/i);
|
|
assert.doesNotMatch(result.error || "", /invalid SSO cookie/i);
|
|
});
|
|
|
|
test("grok-web validator: anti-bot 'Request rejected' 403 maps to IP-reputation guidance (#3474)", async () => {
|
|
__setGrokTlsFetchOverride(async () => {
|
|
return {
|
|
status: 403,
|
|
headers: new Headers(),
|
|
text: "Request rejected by anti-bot rules.",
|
|
body: null,
|
|
};
|
|
});
|
|
|
|
const result = await validateProviderApiKey({ provider: "grok-web", apiKey: "good-cookie" });
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /residential IP|proxy/i);
|
|
// Cookie may be fine — must not claim it is invalid/expired.
|
|
assert.doesNotMatch(result.error || "", /invalid SSO cookie|expired/i);
|
|
});
|
|
|
|
test("grok-web validator: Cloudflare challenge returned with a 403 status maps to IP guidance (#3474)", async () => {
|
|
__setGrokTlsFetchOverride(async () => {
|
|
return {
|
|
status: 403,
|
|
headers: new Headers(),
|
|
text: "<html><title>Just a moment...</title><script>window._cf_chl_opt</script></html>",
|
|
body: null,
|
|
};
|
|
});
|
|
|
|
const result = await validateProviderApiKey({ provider: "grok-web", apiKey: "sso=abc" });
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /residential IP|proxy/i);
|
|
});
|
|
|
|
test("grok-web validator: IP-reputation 403 message does not leak a stack/raw error (Hard Rule #12) (#3474)", async () => {
|
|
__setGrokTlsFetchOverride(async () => {
|
|
return {
|
|
status: 403,
|
|
headers: new Headers(),
|
|
text: "Request rejected by anti-bot rules.\n at GrokWeb.validate (/app/secret/path.ts:42:13)",
|
|
body: null,
|
|
};
|
|
});
|
|
|
|
const result = await validateProviderApiKey({ provider: "grok-web", apiKey: "good-cookie" });
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /residential IP|proxy/i);
|
|
assert.ok(!(result.error || "").includes("at GrokWeb.validate"));
|
|
assert.ok(!(result.error || "").includes("/app/secret/path.ts"));
|
|
assert.ok(!(result.error || "").includes(" at "));
|
|
});
|
|
|
|
test("grok-web validator: structured non-auth 403 (resource error) still surfaces upstream body for maintainers (#3474)", async () => {
|
|
// Distinct from an anti-bot block: a genuine upstream API error (e.g. the probe
|
|
// model was renamed) carries a structured error.message and must NOT be masked
|
|
// by the IP-reputation guidance — the maintainer needs to see the real cause.
|
|
__setGrokTlsFetchOverride(async () => {
|
|
return {
|
|
status: 403,
|
|
headers: new Headers(),
|
|
text: JSON.stringify({ error: { code: 7, message: "Model is not found", details: [] } }),
|
|
body: null,
|
|
};
|
|
});
|
|
|
|
const result = await validateProviderApiKey({ provider: "grok-web", apiKey: "good-cookie" });
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /Grok rejected validation \(403\)/);
|
|
assert.match(result.error || "", /Model is not found/);
|
|
assert.doesNotMatch(result.error || "", /residential IP|proxy/i);
|
|
});
|
|
|
|
test("grok-web validator: auth-shaped 401 keeps the re-paste/re-authenticate guidance (no regression) (#3474)", async () => {
|
|
__setGrokTlsFetchOverride(async () => {
|
|
return { status: 401, headers: new Headers(), text: "Unauthorized", body: null };
|
|
});
|
|
|
|
const result = await validateProviderApiKey({ provider: "grok-web", apiKey: "expired-cookie" });
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /Invalid SSO cookie|re-paste/i);
|
|
assert.doesNotMatch(result.error || "", /residential IP|proxy/i);
|
|
});
|
|
|
|
test("grok-web validator: 403 with credential-rejection body is treated as auth-failed", async () => {
|
|
__setGrokTlsFetchOverride(async () => {
|
|
return {
|
|
status: 403,
|
|
headers: new Headers(),
|
|
text: JSON.stringify({
|
|
error: {
|
|
code: 16,
|
|
message: "Failed to look up session ID. [WKE=unauthenticated:invalid-credentials]",
|
|
details: [],
|
|
},
|
|
}),
|
|
body: null,
|
|
};
|
|
});
|
|
|
|
const result = await validateProviderApiKey({ provider: "grok-web", apiKey: "bad-cookie" });
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /Invalid SSO cookie/i);
|
|
});
|
|
|
|
test("grok-web validator: TLS client unavailable surfaces actionable error", async () => {
|
|
__setGrokTlsFetchOverride(async () => {
|
|
const { TlsClientUnavailableError } = await import("../../open-sse/services/grokTlsClient.ts");
|
|
throw new TlsClientUnavailableError("native binary not found");
|
|
});
|
|
|
|
const result = await validateProviderApiKey({ provider: "grok-web", apiKey: "sso=abc" });
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /TLS impersonation client unavailable/i);
|
|
assert.match(result.error || "", /native binary not found/i);
|
|
});
|
|
|
|
test("grok-web validator: Cloudflare challenge page is detected and reported", async () => {
|
|
__setGrokTlsFetchOverride(async () => {
|
|
return {
|
|
status: 200,
|
|
headers: new Headers(),
|
|
text: "<html><title>Just a moment...</title><script>window._cf_chl_opt</script></html>",
|
|
body: null,
|
|
};
|
|
});
|
|
|
|
const result = await validateProviderApiKey({ provider: "grok-web", apiKey: "sso=abc" });
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /Cloudflare anti-bot/i);
|
|
});
|
|
|
|
// ─── chatgpt-web validator ──────────────────────────────────────────────────
|
|
// Mocks the TLS-impersonating fetch so unit tests don't need the native binding.
|
|
|
|
const { __setTlsFetchOverrideForTesting } =
|
|
await import("../../open-sse/services/chatgptTlsClient.ts");
|
|
|
|
function makeTlsResponse(status: number, body: string, headers: Record<string, string> = {}): any {
|
|
const h = new Headers();
|
|
for (const [k, v] of Object.entries(headers)) h.set(k, v);
|
|
return { status, headers: h, text: body, body: null };
|
|
}
|
|
|
|
test.afterEach(() => {
|
|
__setTlsFetchOverrideForTesting(null);
|
|
});
|
|
|
|
test("chatgpt-web validator: accepts a valid session response with accessToken", async () => {
|
|
let captured: { url: string; opts: any } | null = null;
|
|
__setTlsFetchOverrideForTesting(async (url, opts) => {
|
|
captured = { url, opts };
|
|
return makeTlsResponse(
|
|
200,
|
|
JSON.stringify({ accessToken: "tok-abc", expires: "2030-01-01T00:00:00Z" }),
|
|
{ "content-type": "application/json" }
|
|
);
|
|
});
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "chatgpt-web",
|
|
apiKey: "__Secure-next-auth.session-token=eyJSESSION",
|
|
});
|
|
|
|
assert.equal(result.valid, true);
|
|
assert.equal(captured?.url, "https://chatgpt.com/api/auth/session");
|
|
assert.equal(
|
|
(captured?.opts.headers as Record<string, string>).Cookie,
|
|
"__Secure-next-auth.session-token=eyJSESSION"
|
|
);
|
|
});
|
|
|
|
test("chatgpt-web validator: prepends session-token name to bare values", async () => {
|
|
let capturedCookie = "";
|
|
__setTlsFetchOverrideForTesting(async (_url, opts) => {
|
|
capturedCookie = (opts.headers as Record<string, string>).Cookie || "";
|
|
return makeTlsResponse(200, JSON.stringify({ accessToken: "tok" }), {
|
|
"content-type": "application/json",
|
|
});
|
|
});
|
|
|
|
await validateProviderApiKey({ provider: "chatgpt-web", apiKey: "eyJBARE" });
|
|
assert.equal(capturedCookie, "__Secure-next-auth.session-token=eyJBARE");
|
|
});
|
|
|
|
test("chatgpt-web validator: passes full DevTools cookie blob through verbatim", async () => {
|
|
let capturedCookie = "";
|
|
__setTlsFetchOverrideForTesting(async (_url, opts) => {
|
|
capturedCookie = (opts.headers as Record<string, string>).Cookie || "";
|
|
return makeTlsResponse(200, JSON.stringify({ accessToken: "tok" }), {
|
|
"content-type": "application/json",
|
|
});
|
|
});
|
|
|
|
const blob =
|
|
"Cookie: oai-did=foo; __Secure-next-auth.session-token.0=eyJchunk0; __Secure-next-auth.session-token.1=eyJchunk1; cf_clearance=cf123;";
|
|
await validateProviderApiKey({ provider: "chatgpt-web", apiKey: blob });
|
|
assert.equal(
|
|
capturedCookie,
|
|
"oai-did=foo; __Secure-next-auth.session-token.0=eyJchunk0; __Secure-next-auth.session-token.1=eyJchunk1; cf_clearance=cf123;"
|
|
);
|
|
});
|
|
|
|
test("chatgpt-web validator: 401 without cf-mitigated → invalid session cookie", async () => {
|
|
__setTlsFetchOverrideForTesting(async () =>
|
|
makeTlsResponse(401, JSON.stringify({ error: "unauthorized" }), {
|
|
"content-type": "application/json",
|
|
})
|
|
);
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "chatgpt-web",
|
|
apiKey: "stale-token",
|
|
});
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /Invalid ChatGPT session cookie/i);
|
|
});
|
|
|
|
test("chatgpt-web validator: 403 with cf-mitigated header → Cloudflare hint", async () => {
|
|
__setTlsFetchOverrideForTesting(async () =>
|
|
makeTlsResponse(403, "<html>Just a moment...</html>", {
|
|
"content-type": "text/html",
|
|
"cf-mitigated": "challenge",
|
|
})
|
|
);
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "chatgpt-web",
|
|
apiKey: "good-but-no-cf-cookies",
|
|
});
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /Cloudflare blocked the validator/i);
|
|
});
|
|
|
|
test("chatgpt-web validator: 200 without accessToken → session expired", async () => {
|
|
__setTlsFetchOverrideForTesting(async () =>
|
|
makeTlsResponse(200, JSON.stringify({}), { "content-type": "application/json" })
|
|
);
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "chatgpt-web",
|
|
apiKey: "expired-token",
|
|
});
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /session expired/i);
|
|
});
|
|
|
|
test("chatgpt-web validator: 5xx → ChatGPT unavailable", async () => {
|
|
__setTlsFetchOverrideForTesting(async () =>
|
|
makeTlsResponse(503, "service unavailable", { "content-type": "text/plain" })
|
|
);
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "chatgpt-web",
|
|
apiKey: "any-token",
|
|
});
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /ChatGPT unavailable \(503\)/);
|
|
});
|
|
|
|
test("chatgpt-web validator: 200 non-JSON content-type surfaces a cookie hint", async () => {
|
|
__setTlsFetchOverrideForTesting(async () =>
|
|
makeTlsResponse(200, "<html>blocked</html>", {
|
|
"content-type": "text/html",
|
|
"cf-ray": "ray-123",
|
|
})
|
|
);
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "chatgpt-web",
|
|
apiKey: "any-token",
|
|
});
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /non-JSON.*text\/html.*cf-ray=ray-123/i);
|
|
});
|
|
|
|
test("chatgpt-web validator: TlsClientUnavailableError surfaces a clear message", async () => {
|
|
const { TlsClientUnavailableError } = await import("../../open-sse/services/chatgptTlsClient.ts");
|
|
__setTlsFetchOverrideForTesting(async () => {
|
|
throw new TlsClientUnavailableError("native binding failed to load");
|
|
});
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "chatgpt-web",
|
|
apiKey: "any-token",
|
|
});
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /chatgpt-web requires this/i);
|
|
});
|
|
|
|
test("search provider validators cover success, client errors, server errors and custom user agent injection", async () => {
|
|
const calls = [];
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
calls.push({ url: String(url), init });
|
|
const target = String(url);
|
|
if (target.match(/search\.brave\.com/i)) {
|
|
return new Response(JSON.stringify({ results: [] }), { status: 200 });
|
|
}
|
|
if (target.match(/api\.exa\.ai/i)) {
|
|
return new Response(JSON.stringify({ error: "bad key" }), { status: 403 });
|
|
}
|
|
if (target.match(/api\.tavily\.com/i)) {
|
|
return new Response(JSON.stringify({ error: "server" }), { status: 503 });
|
|
}
|
|
if (target.match(/api\.perplexity\.ai/i)) {
|
|
throw new Error("perplexity offline");
|
|
}
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const brave = await validateProviderApiKey({
|
|
provider: "brave-search",
|
|
apiKey: "brave-key",
|
|
providerSpecificData: { customUserAgent: "SearchSuite/1.0" },
|
|
});
|
|
const exa = await validateProviderApiKey({ provider: "exa-search", apiKey: "exa-key" });
|
|
const tavily = await validateProviderApiKey({ provider: "tavily-search", apiKey: "tv-key" });
|
|
const perplexity = await validateProviderApiKey({
|
|
provider: "perplexity-search",
|
|
apiKey: "px-key",
|
|
});
|
|
|
|
assert.equal(brave.valid, true);
|
|
assert.equal(exa.error, "Invalid API key");
|
|
assert.equal(tavily.error, "Validation failed: 503");
|
|
assert.equal(perplexity.error, "perplexity offline");
|
|
assert.equal(calls[0].init.headers["User-Agent"], "SearchSuite/1.0");
|
|
});
|
|
|
|
test("extended search provider validators cover Google PSE, Linkup, SearchAPI, You.com and SearXNG", async () => {
|
|
const originalAllowPrivateProviderUrls = process.env.OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS;
|
|
process.env.OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS = "true";
|
|
const calls = [];
|
|
try {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
calls.push({ url: String(url), init });
|
|
const target = String(url);
|
|
if (target.startsWith("https://www.googleapis.com/customsearch/v1")) {
|
|
return new Response(JSON.stringify({ items: [] }), { status: 200 });
|
|
}
|
|
if (target.startsWith("https://api.linkup.so/v1/search")) {
|
|
return new Response(JSON.stringify({ error: "bad request" }), { status: 400 });
|
|
}
|
|
if (target.startsWith("https://www.searchapi.io/api/v1/search")) {
|
|
return new Response(JSON.stringify({ organic_results: [] }), { status: 200 });
|
|
}
|
|
if (target.startsWith("https://ydc-index.io/v1/search")) {
|
|
return new Response(JSON.stringify({ results: { web: [] } }), { status: 200 });
|
|
}
|
|
if (target.startsWith("http://localhost:9999/search")) {
|
|
return new Response(JSON.stringify({ results: [] }), { status: 200 });
|
|
}
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const google = await validateProviderApiKey({
|
|
provider: "google-pse-search",
|
|
apiKey: "google-key",
|
|
providerSpecificData: { cx: "engine-id" },
|
|
});
|
|
const linkup = await validateProviderApiKey({
|
|
provider: "linkup-search",
|
|
apiKey: "linkup-key",
|
|
});
|
|
const searchapi = await validateProviderApiKey({
|
|
provider: "searchapi-search",
|
|
apiKey: "searchapi-key",
|
|
});
|
|
const youcom = await validateProviderApiKey({
|
|
provider: "youcom-search",
|
|
apiKey: "you-key",
|
|
});
|
|
const searxng = await validateProviderApiKey({
|
|
provider: "searxng-search",
|
|
providerSpecificData: { baseUrl: "http://localhost:9999/search" },
|
|
});
|
|
|
|
assert.equal(google.valid, true);
|
|
assert.equal(linkup.valid, true);
|
|
assert.equal(searchapi.valid, true);
|
|
assert.equal(youcom.valid, true);
|
|
assert.equal(searxng.valid, true);
|
|
assert.match(calls[0].url, /cx=engine-id/);
|
|
assert.equal(calls[1].init.headers.Authorization, "Bearer linkup-key");
|
|
assert.match(calls[2].url, /api_key=searchapi-key/);
|
|
assert.equal(calls[3].init.headers["X-API-Key"], "you-key");
|
|
} finally {
|
|
if (originalAllowPrivateProviderUrls === undefined) {
|
|
delete process.env.OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS;
|
|
} else {
|
|
process.env.OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS = originalAllowPrivateProviderUrls;
|
|
}
|
|
}
|
|
});
|
|
|
|
test("google PSE validator requires cx", async () => {
|
|
const result = await validateProviderApiKey({
|
|
provider: "google-pse-search",
|
|
apiKey: "google-key",
|
|
});
|
|
|
|
assert.equal(result.valid, false);
|
|
assert.equal(result.error, "Programmable Search Engine ID (cx) is required");
|
|
});
|
|
|
|
test("Maritalk validates with Key auth against the models endpoint", async () => {
|
|
const calls = [];
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
calls.push({ url: String(url), init });
|
|
assert.equal(String(url), "https://chat.maritaca.ai/api/models");
|
|
assert.equal(init.headers.Authorization, "Key maritalk-key");
|
|
return new Response(JSON.stringify({ data: [{ id: "sabia-4" }] }), {
|
|
status: 200,
|
|
});
|
|
};
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "maritalk",
|
|
apiKey: "maritalk-key",
|
|
});
|
|
|
|
assert.equal(result.valid, true);
|
|
assert.equal(result.method, "maritalk_models");
|
|
assert.equal(calls.length, 1);
|
|
});
|
|
|
|
test("Maritalk falls back to chat probe when the models endpoint is unreachable", async () => {
|
|
const calls = [];
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
calls.push({ url: String(url), init });
|
|
|
|
if (String(url) === "https://chat.maritaca.ai/api/models") {
|
|
return new Response(JSON.stringify({ error: "not found" }), { status: 404 });
|
|
}
|
|
|
|
assert.equal(String(url), "https://chat.maritaca.ai/api/chat/completions");
|
|
assert.equal(init.headers.Authorization, "Key maritalk-key");
|
|
const body = JSON.parse(String(init.body));
|
|
assert.equal(body.model, "sabia-4");
|
|
return new Response(JSON.stringify({ choices: [{ message: { content: "ok" } }] }), {
|
|
status: 200,
|
|
});
|
|
};
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "maritalk",
|
|
apiKey: "maritalk-key",
|
|
});
|
|
|
|
assert.equal(result.valid, true);
|
|
assert.equal(calls.length, 2);
|
|
});
|
|
|
|
test("Maritalk treats a rate-limited models probe as valid credentials", async () => {
|
|
const calls = [];
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
calls.push({ url: String(url), init });
|
|
assert.equal(String(url), "https://chat.maritaca.ai/api/models");
|
|
assert.equal(init.headers.Authorization, "Key maritalk-key");
|
|
return new Response(JSON.stringify({ error: "rate limited" }), {
|
|
status: 429,
|
|
});
|
|
};
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "maritalk",
|
|
apiKey: "maritalk-key",
|
|
});
|
|
|
|
assert.equal(result.valid, true);
|
|
assert.equal(result.warning, "Rate limited, but credentials are valid");
|
|
assert.equal(calls.length, 1);
|
|
});
|
|
|
|
test("local OpenAI-style providers validate without sending Authorization when apiKey is blank", async () => {
|
|
const originalAllowPrivateProviderUrls = process.env.OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS;
|
|
process.env.OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS = "true";
|
|
const calls = [];
|
|
|
|
try {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
calls.push({ url: String(url), headers: init.headers || {} });
|
|
return new Response(JSON.stringify({ data: [{ id: "local-model" }] }), { status: 200 });
|
|
};
|
|
|
|
const lmStudio = await validateProviderApiKey({
|
|
provider: "lm-studio",
|
|
providerSpecificData: { baseUrl: "http://localhost:1234/v1" },
|
|
});
|
|
const vllm = await validateProviderApiKey({
|
|
provider: "vllm",
|
|
providerSpecificData: { baseUrl: "http://localhost:8000/v1" },
|
|
});
|
|
const lemonade = await validateProviderApiKey({
|
|
provider: "lemonade",
|
|
providerSpecificData: { baseUrl: "http://localhost:13305/api/v1" },
|
|
});
|
|
const llamaCpp = await validateProviderApiKey({
|
|
provider: "llama-cpp",
|
|
providerSpecificData: { baseUrl: "http://127.0.0.1:8080/v1" },
|
|
});
|
|
|
|
assert.equal(lmStudio.valid, true);
|
|
assert.equal(vllm.valid, true);
|
|
assert.equal(lemonade.valid, true);
|
|
assert.equal(llamaCpp.valid, true);
|
|
assert.deepEqual(
|
|
calls.map((call) => call.url),
|
|
[
|
|
"http://localhost:1234/v1/models",
|
|
"http://localhost:8000/v1/models",
|
|
"http://localhost:13305/api/v1/models",
|
|
"http://127.0.0.1:8080/v1/models",
|
|
]
|
|
);
|
|
assert.equal(calls[0].headers.Authorization, undefined);
|
|
assert.equal(calls[1].headers.Authorization, undefined);
|
|
assert.equal(calls[2].headers.Authorization, undefined);
|
|
assert.equal(calls[3].headers.Authorization, undefined);
|
|
} finally {
|
|
if (originalAllowPrivateProviderUrls === undefined) {
|
|
delete process.env.OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS;
|
|
} else {
|
|
process.env.OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS = originalAllowPrivateProviderUrls;
|
|
}
|
|
}
|
|
});
|
|
|
|
test("OpenAI-compatible validator covers /responses mode and final ping fallback", async () => {
|
|
const calls = [];
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
calls.push({ url: String(url), method: init.method || "GET" });
|
|
if (String(url).endsWith("/models")) {
|
|
return new Response(JSON.stringify({ error: "no models" }), { status: 500 });
|
|
}
|
|
if (String(url).endsWith("/responses")) {
|
|
return new Response(JSON.stringify({ id: "resp_123" }), { status: 200 });
|
|
}
|
|
if (String(url) === "https://openai-like.example.com/v1") {
|
|
return new Response("ok", { status: 418 });
|
|
}
|
|
throw new Error(`unexpected fetch: ${url}`);
|
|
};
|
|
|
|
const responsesResult = await validateProviderApiKey({
|
|
provider: "openai-compatible-responses",
|
|
apiKey: "sk-test",
|
|
providerSpecificData: {
|
|
baseUrl: "https://openai-like.example.com/v1",
|
|
apiType: "responses",
|
|
validationModelId: "gpt-test",
|
|
},
|
|
});
|
|
|
|
globalThis.fetch = async (url) => {
|
|
if (String(url).endsWith("/models")) {
|
|
return new Response(JSON.stringify({ error: "server" }), { status: 500 });
|
|
}
|
|
if (String(url).endsWith("/chat/completions")) {
|
|
throw new Error("chat probe offline");
|
|
}
|
|
return new Response("teapot", { status: 418 });
|
|
};
|
|
|
|
const pingFallback = await validateProviderApiKey({
|
|
provider: "openai-compatible-ping-fallback",
|
|
apiKey: "sk-test",
|
|
providerSpecificData: {
|
|
baseUrl: "https://openai-like.example.com/v1",
|
|
validationModelId: "gpt-test",
|
|
},
|
|
});
|
|
|
|
assert.equal(responsesResult.valid, true);
|
|
assert.equal(responsesResult.method, "chat_completions");
|
|
assert.deepEqual(
|
|
calls.map((call) => call.url),
|
|
["https://openai-like.example.com/v1/models", "https://openai-like.example.com/v1/responses"]
|
|
);
|
|
assert.equal(pingFallback.valid, true);
|
|
assert.equal(pingFallback.error, null);
|
|
});
|
|
|
|
test("Anthropic-compatible and Claude Code compatible validators cover direct success and bridge fallbacks", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
if (target.match(/anthropic-compatible\.example\.com/i) && init.method === "GET") {
|
|
return new Response(JSON.stringify({ data: [] }), { status: 200 });
|
|
}
|
|
if (target.match(/cc-compatible\.example\.com/i) && init.method === "GET") {
|
|
return new Response(JSON.stringify({ error: "bridge unavailable" }), { status: 500 });
|
|
}
|
|
if (target.match(/cc-compatible\.example\.com/i) && init.method === "POST") {
|
|
return new Response(JSON.stringify({ error: "rate limited" }), { status: 429 });
|
|
}
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const anthropic = await validateProviderApiKey({
|
|
provider: "anthropic-compatible-direct",
|
|
apiKey: "sk-anthropic",
|
|
providerSpecificData: {
|
|
baseUrl: "https://anthropic-compatible.example.com/v1/messages",
|
|
modelsPath: "/custom-models",
|
|
},
|
|
});
|
|
|
|
const ccRateLimited = await validateClaudeCodeCompatibleProvider({
|
|
apiKey: "sk-cc",
|
|
providerSpecificData: {
|
|
baseUrl: "https://cc-compatible.example.com/v1/messages",
|
|
validationModelId: "claude-bridge-test",
|
|
},
|
|
});
|
|
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
if (init.method === "GET") {
|
|
return new Response(JSON.stringify({ error: "bridge unavailable" }), { status: 500 });
|
|
}
|
|
return new Response(JSON.stringify({ error: "bad gateway" }), { status: 502 });
|
|
};
|
|
|
|
const ccFailure = await validateClaudeCodeCompatibleProvider({
|
|
apiKey: "sk-cc",
|
|
providerSpecificData: {
|
|
baseUrl: "https://cc-compatible.example.com/v1/messages",
|
|
},
|
|
});
|
|
|
|
assert.equal(anthropic.valid, true);
|
|
assert.equal(ccRateLimited.valid, true);
|
|
assert.equal(ccRateLimited.method, "cc_bridge_request");
|
|
assert.match(ccRateLimited.warning, /Rate limited/i);
|
|
assert.equal(ccFailure.valid, false);
|
|
assert.equal(ccFailure.error, "Validation failed: 502");
|
|
});
|
|
|
|
test("Claude Code compatible validator rejects missing base URL and bridge auth failures", async () => {
|
|
const missingBase = await validateClaudeCodeCompatibleProvider({
|
|
apiKey: "sk-cc",
|
|
providerSpecificData: {},
|
|
});
|
|
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
if (init.method === "GET") {
|
|
throw new Error("models offline");
|
|
}
|
|
return new Response(JSON.stringify({ error: "forbidden" }), { status: 403 });
|
|
};
|
|
|
|
const invalidKey = await validateClaudeCodeCompatibleProvider({
|
|
apiKey: "sk-cc",
|
|
providerSpecificData: {
|
|
baseUrl: "https://cc-compatible.example.com/v1/messages",
|
|
},
|
|
});
|
|
|
|
assert.equal(missingBase.error, "No base URL configured for CC Compatible provider");
|
|
assert.equal(invalidKey.error, "Invalid API key");
|
|
});
|
|
|
|
test("registry providers cover remaining OpenAI-like and Claude-like validation branches", async () => {
|
|
const calls = [];
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
calls.push({ url: String(url), method: init.method || "GET", headers: init.headers || {} });
|
|
const target = String(url);
|
|
|
|
if (target === "https://api.openai.com/v1/models") {
|
|
return new Response(JSON.stringify({ data: [{ id: "gpt-4o-mini" }] }), { status: 200 });
|
|
}
|
|
if (target === "https://api.anthropic.com/v1/messages?beta=true") {
|
|
return new Response(JSON.stringify({ id: "msg_123" }), { status: 200 });
|
|
}
|
|
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const openaiModels = await validateProviderApiKey({ provider: "openai", apiKey: "sk-openai" });
|
|
const claudeSuccess = await validateProviderApiKey({ provider: "claude", apiKey: "sk-claude" });
|
|
|
|
globalThis.fetch = async (url) => {
|
|
if (String(url) === "https://api.openai.com/v1/models") {
|
|
return new Response(JSON.stringify({ error: "server" }), { status: 500 });
|
|
}
|
|
if (String(url) === "https://api.openai.com/v1/chat/completions") {
|
|
return new Response(JSON.stringify({ error: "method not allowed" }), { status: 405 });
|
|
}
|
|
throw new Error(`unexpected fetch: ${url}`);
|
|
};
|
|
const openaiUnsupported = await validateProviderApiKey({
|
|
provider: "openai",
|
|
apiKey: "sk-openai",
|
|
});
|
|
|
|
globalThis.fetch = async (url) => {
|
|
if (String(url) === "https://api.openai.com/v1/models") {
|
|
return new Response(JSON.stringify({ error: "server" }), { status: 500 });
|
|
}
|
|
if (String(url) === "https://api.openai.com/v1/chat/completions") {
|
|
return new Response(JSON.stringify({ error: "unprocessable" }), { status: 422 });
|
|
}
|
|
throw new Error(`unexpected fetch: ${url}`);
|
|
};
|
|
const openaiInference = await validateProviderApiKey({ provider: "openai", apiKey: "sk-openai" });
|
|
|
|
globalThis.fetch = async (url) => {
|
|
if (String(url) === "https://api.openai.com/v1/models") {
|
|
return new Response(JSON.stringify({ error: "server" }), { status: 500 });
|
|
}
|
|
if (String(url) === "https://api.openai.com/v1/chat/completions") {
|
|
return new Response(JSON.stringify({ error: "server" }), { status: 502 });
|
|
}
|
|
throw new Error(`unexpected fetch: ${url}`);
|
|
};
|
|
const openaiUnavailable = await validateProviderApiKey({
|
|
provider: "openai",
|
|
apiKey: "sk-openai",
|
|
});
|
|
|
|
globalThis.fetch = async () =>
|
|
new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
|
|
const claudeInvalid = await validateProviderApiKey({ provider: "claude", apiKey: "sk-claude" });
|
|
|
|
globalThis.fetch = async () => {
|
|
throw new Error("anthropic offline");
|
|
};
|
|
const claudeOffline = await validateProviderApiKey({ provider: "claude", apiKey: "sk-claude" });
|
|
|
|
assert.equal(openaiModels.valid, true);
|
|
assert.equal(openaiModels.error, null);
|
|
assert.equal(claudeSuccess.valid, true);
|
|
assert.equal(openaiUnsupported.error, "Provider validation endpoint not supported");
|
|
assert.equal(openaiInference.valid, true);
|
|
assert.equal(openaiInference.error, null);
|
|
assert.equal(openaiUnavailable.error, "Provider unavailable (502)");
|
|
assert.equal(claudeInvalid.error, "Invalid API key");
|
|
assert.equal(claudeOffline.error, "anthropic offline");
|
|
assert.equal(calls[1].headers["x-api-key"], "sk-claude");
|
|
});
|
|
|
|
test("specialty validators cover remaining status branches for Deepgram, AssemblyAI, ElevenLabs, Inworld, Bailian and LongCat", async () => {
|
|
globalThis.fetch = async (url) => {
|
|
const target = String(url);
|
|
if (target.match(/deepgram/i)) {
|
|
return new Response(JSON.stringify({ error: "server" }), { status: 500 });
|
|
}
|
|
if (target.match(/assemblyai/i)) {
|
|
return new Response(JSON.stringify({ transcripts: [] }), { status: 200 });
|
|
}
|
|
if (target.match(/elevenlabs/i)) {
|
|
return new Response(JSON.stringify({ error: "forbidden" }), { status: 403 });
|
|
}
|
|
if (target.match(/inworld/i)) {
|
|
throw new Error("inworld offline");
|
|
}
|
|
if (target.match(/dashscope\.aliyuncs\.com/i)) {
|
|
return new Response(JSON.stringify({ error: "server" }), { status: 500 });
|
|
}
|
|
if (target.match(/longcat/i)) {
|
|
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
|
|
}
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const deepgram = await validateProviderApiKey({ provider: "deepgram", apiKey: "dg-key" });
|
|
const assembly = await validateProviderApiKey({ provider: "assemblyai", apiKey: "aa-key" });
|
|
const eleven = await validateProviderApiKey({ provider: "elevenlabs", apiKey: "el-key" });
|
|
const inworld = await validateProviderApiKey({ provider: "inworld", apiKey: "iw-key" });
|
|
const bailian = await validateProviderApiKey({
|
|
provider: "bailian-coding-plan",
|
|
apiKey: "bailian-key",
|
|
providerSpecificData: {
|
|
baseUrl: "https://coding-intl.dashscope.aliyuncs.com/apps/anthropic/v1/messages",
|
|
},
|
|
});
|
|
const longcatInvalid = await validateProviderApiKey({ provider: "longcat", apiKey: "lc-key" });
|
|
|
|
globalThis.fetch = async (url) => {
|
|
if (String(url).match(/elevenlabs/i)) {
|
|
throw new Error("elevenlabs offline");
|
|
}
|
|
if (String(url).match(/longcat/i)) {
|
|
return new Response(JSON.stringify({ error: "unprocessable" }), { status: 422 });
|
|
}
|
|
throw new Error(`unexpected fetch: ${url}`);
|
|
};
|
|
|
|
const elevenOffline = await validateProviderApiKey({ provider: "elevenlabs", apiKey: "el-key" });
|
|
const longcatValid = await validateProviderApiKey({ provider: "longcat", apiKey: "lc-key" });
|
|
|
|
assert.equal(deepgram.error, "Validation failed: 500");
|
|
assert.equal(assembly.valid, true);
|
|
assert.equal(eleven.error, "Invalid API key");
|
|
assert.equal(inworld.error, "inworld offline");
|
|
assert.equal(bailian.error, "Validation failed: 500");
|
|
assert.equal(longcatInvalid.error, "Invalid API key");
|
|
assert.equal(elevenOffline.error, "elevenlabs offline");
|
|
assert.equal(longcatValid.valid, true);
|
|
});
|
|
|
|
test("specialty validators cover Heroku, Databricks, Snowflake and GigaChat success paths", async () => {
|
|
const seen = [];
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
seen.push({ url: target, headers: init.headers || {} });
|
|
|
|
if (target === "https://ngw.devices.sberbank.ru:9443/api/v2/oauth") {
|
|
assert.equal(init.headers.Authorization, "Basic gigachat-basic-creds");
|
|
return new Response(
|
|
JSON.stringify({
|
|
tok: "gigachat-access-token",
|
|
exp: Date.now() + 60 * 60 * 1000,
|
|
}),
|
|
{ status: 200 }
|
|
);
|
|
}
|
|
if (target === "https://us.inference.heroku.com/v1/chat/completions") {
|
|
assert.equal(init.headers.Authorization, "Bearer heroku-key");
|
|
return new Response(JSON.stringify({ error: "bad request" }), { status: 400 });
|
|
}
|
|
if (
|
|
target ===
|
|
"https://adb-1234567890123456.7.azuredatabricks.net/serving-endpoints/chat/completions"
|
|
) {
|
|
assert.equal(init.headers.Authorization, "Bearer databricks-key");
|
|
return new Response(JSON.stringify({ error: "unprocessable" }), { status: 422 });
|
|
}
|
|
if (target === "https://account.snowflakecomputing.com/api/v2/cortex/inference:complete") {
|
|
assert.equal(init.headers.Authorization, "Bearer snowflake-token");
|
|
assert.equal(
|
|
init.headers["X-Snowflake-Authorization-Token-Type"],
|
|
"PROGRAMMATIC_ACCESS_TOKEN"
|
|
);
|
|
return new Response(JSON.stringify({ error: "bad request" }), { status: 400 });
|
|
}
|
|
if (target === "https://gigachat.devices.sberbank.ru/api/v1/chat/completions") {
|
|
assert.equal(init.headers.Authorization, "Bearer gigachat-access-token");
|
|
return new Response(JSON.stringify({ error: "bad request" }), { status: 400 });
|
|
}
|
|
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const heroku = await validateProviderApiKey({
|
|
provider: "heroku",
|
|
apiKey: "heroku-key",
|
|
providerSpecificData: { baseUrl: "https://us.inference.heroku.com" },
|
|
});
|
|
const databricks = await validateProviderApiKey({
|
|
provider: "databricks",
|
|
apiKey: "databricks-key",
|
|
providerSpecificData: {
|
|
baseUrl: "https://adb-1234567890123456.7.azuredatabricks.net/serving-endpoints",
|
|
},
|
|
});
|
|
const snowflake = await validateProviderApiKey({
|
|
provider: "snowflake",
|
|
apiKey: "pat/snowflake-token",
|
|
providerSpecificData: { baseUrl: "https://account.snowflakecomputing.com" },
|
|
});
|
|
const gigachat = await validateProviderApiKey({
|
|
provider: "gigachat",
|
|
apiKey: "gigachat-basic-creds",
|
|
});
|
|
|
|
assert.equal(heroku.valid, true);
|
|
assert.equal(databricks.valid, true);
|
|
assert.equal(snowflake.valid, true);
|
|
assert.equal(gigachat.valid, true);
|
|
assert.equal(
|
|
seen.some((call) => call.url === "https://ngw.devices.sberbank.ru:9443/api/v2/oauth"),
|
|
true
|
|
);
|
|
});
|
|
|
|
test("specialty validators surface missing base URLs and invalid auth for Heroku, Databricks, Snowflake and GigaChat", async () => {
|
|
const missingHerokuBase = await validateProviderApiKey({
|
|
provider: "heroku",
|
|
apiKey: "heroku-key",
|
|
providerSpecificData: {},
|
|
});
|
|
const missingDatabricksBase = await validateProviderApiKey({
|
|
provider: "databricks",
|
|
apiKey: "databricks-key",
|
|
providerSpecificData: {},
|
|
});
|
|
const missingSnowflakeBase = await validateProviderApiKey({
|
|
provider: "snowflake",
|
|
apiKey: "snowflake-key",
|
|
providerSpecificData: {},
|
|
});
|
|
|
|
globalThis.fetch = async (url) => {
|
|
const target = String(url);
|
|
if (target === "https://ngw.devices.sberbank.ru:9443/api/v2/oauth") {
|
|
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
|
|
}
|
|
if (target === "https://us.inference.heroku.com/v1/chat/completions") {
|
|
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
|
|
}
|
|
if (
|
|
target ===
|
|
"https://adb-1234567890123456.7.azuredatabricks.net/serving-endpoints/chat/completions"
|
|
) {
|
|
return new Response(JSON.stringify({ error: "forbidden" }), { status: 403 });
|
|
}
|
|
if (target === "https://account.snowflakecomputing.com/api/v2/cortex/inference:complete") {
|
|
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
|
|
}
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const herokuInvalid = await validateProviderApiKey({
|
|
provider: "heroku",
|
|
apiKey: "heroku-key",
|
|
providerSpecificData: { baseUrl: "https://us.inference.heroku.com" },
|
|
});
|
|
const databricksInvalid = await validateProviderApiKey({
|
|
provider: "databricks",
|
|
apiKey: "databricks-key",
|
|
providerSpecificData: {
|
|
baseUrl: "https://adb-1234567890123456.7.azuredatabricks.net/serving-endpoints",
|
|
},
|
|
});
|
|
const snowflakeInvalid = await validateProviderApiKey({
|
|
provider: "snowflake",
|
|
apiKey: "snowflake-key",
|
|
providerSpecificData: { baseUrl: "https://account.snowflakecomputing.com" },
|
|
});
|
|
const gigachatInvalid = await validateProviderApiKey({
|
|
provider: "gigachat",
|
|
apiKey: "gigachat-basic-creds-invalid",
|
|
});
|
|
|
|
assert.equal(missingHerokuBase.error, "Missing base URL");
|
|
assert.equal(missingDatabricksBase.error, "Missing base URL");
|
|
assert.equal(missingSnowflakeBase.error, "Missing base URL");
|
|
assert.equal(herokuInvalid.error, "Invalid API key");
|
|
assert.equal(databricksInvalid.error, "Invalid API key");
|
|
assert.equal(snowflakeInvalid.error, "Invalid API key");
|
|
assert.equal(gigachatInvalid.error, "Invalid API key");
|
|
});
|
|
|
|
test("specialty validator accepts DataRobot gateway and deployment credentials", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
|
|
if (target === "https://app.datarobot.com/genai/llmgw/catalog/") {
|
|
assert.equal((init.headers as Record<string, string>).Authorization, "Bearer dr-key");
|
|
return new Response(
|
|
JSON.stringify({
|
|
data: [{ model: "azure/gpt-5-mini-2025-08-07", isActive: true }],
|
|
}),
|
|
{ status: 200 }
|
|
);
|
|
}
|
|
|
|
if (
|
|
target ===
|
|
"https://app.datarobot.com/api/v2/deployments/65f5b2b7c8f8c4b257e0d123/chat/completions"
|
|
) {
|
|
assert.equal((init.headers as Record<string, string>).Authorization, "Bearer dr-deploy-key");
|
|
const body = JSON.parse(String(init.body));
|
|
assert.equal(body.model, "datarobot-deployed-llm");
|
|
return new Response(JSON.stringify({ error: "bad request" }), { status: 400 });
|
|
}
|
|
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const gateway = await validateProviderApiKey({
|
|
provider: "datarobot",
|
|
apiKey: "dr-key",
|
|
});
|
|
const deployment = await validateProviderApiKey({
|
|
provider: "datarobot",
|
|
apiKey: "dr-deploy-key",
|
|
providerSpecificData: {
|
|
baseUrl: "https://app.datarobot.com/api/v2/deployments/65f5b2b7c8f8c4b257e0d123",
|
|
},
|
|
});
|
|
|
|
assert.equal(gateway.valid, true);
|
|
assert.equal(deployment.valid, true);
|
|
});
|
|
|
|
test("specialty validator rejects invalid DataRobot credentials", async () => {
|
|
globalThis.fetch = async (url) => {
|
|
const target = String(url);
|
|
|
|
if (target === "https://app.datarobot.com/genai/llmgw/catalog/") {
|
|
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
|
|
}
|
|
|
|
if (
|
|
target ===
|
|
"https://app.datarobot.com/api/v2/deployments/65f5b2b7c8f8c4b257e0d123/chat/completions"
|
|
) {
|
|
return new Response(JSON.stringify({ error: "forbidden" }), { status: 403 });
|
|
}
|
|
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const gateway = await validateProviderApiKey({
|
|
provider: "datarobot",
|
|
apiKey: "dr-key",
|
|
});
|
|
const deployment = await validateProviderApiKey({
|
|
provider: "datarobot",
|
|
apiKey: "dr-deploy-key",
|
|
providerSpecificData: {
|
|
baseUrl: "https://app.datarobot.com/api/v2/deployments/65f5b2b7c8f8c4b257e0d123",
|
|
},
|
|
});
|
|
|
|
assert.equal(gateway.error, "Invalid API key");
|
|
assert.equal(deployment.error, "Invalid API key");
|
|
});
|
|
|
|
test("specialty validators accept watsonx, OCI and SAP enterprise gateways", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
|
|
if (target === "https://ca-tor.ml.cloud.ibm.com/ml/gateway/v1/models") {
|
|
assert.equal((init.headers as Record<string, string>).Authorization, "Bearer watsonx-key");
|
|
return new Response(JSON.stringify({ data: [] }), { status: 200 });
|
|
}
|
|
|
|
if (
|
|
target === "https://inference.generativeai.us-chicago-1.oci.oraclecloud.com/openai/v1/models"
|
|
) {
|
|
const headers = init.headers as Record<string, string>;
|
|
assert.equal(headers.Authorization, "Bearer oci-key");
|
|
assert.equal(headers["OpenAI-Project"], "ocid1.generativeaiproject.oc1.us-chicago-1.demo");
|
|
return new Response(JSON.stringify({ data: [] }), { status: 200 });
|
|
}
|
|
|
|
if (target === "https://sap.example.com/v2/lm/scenarios/foundation-models/models") {
|
|
const headers = init.headers as Record<string, string>;
|
|
assert.equal(headers.Authorization, "Bearer sap-key");
|
|
assert.equal(headers["AI-Resource-Group"], "shared");
|
|
return new Response(JSON.stringify({ resources: [] }), { status: 200 });
|
|
}
|
|
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const watsonx = await validateProviderApiKey({
|
|
provider: "watsonx",
|
|
apiKey: "watsonx-key",
|
|
providerSpecificData: { baseUrl: "https://ca-tor.ml.cloud.ibm.com" },
|
|
});
|
|
const oci = await validateProviderApiKey({
|
|
provider: "oci",
|
|
apiKey: "oci-key",
|
|
providerSpecificData: {
|
|
baseUrl: "https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
|
|
projectId: "ocid1.generativeaiproject.oc1.us-chicago-1.demo",
|
|
},
|
|
});
|
|
const sap = await validateProviderApiKey({
|
|
provider: "sap",
|
|
apiKey: "sap-key",
|
|
providerSpecificData: {
|
|
baseUrl: "https://sap.example.com/v2/lm/deployments/demo-deployment",
|
|
resourceGroup: "shared",
|
|
},
|
|
});
|
|
|
|
assert.equal(watsonx.valid, true);
|
|
assert.equal(watsonx.method, "watsonx_models");
|
|
assert.equal(oci.valid, true);
|
|
assert.equal(oci.method, "oci_models");
|
|
assert.equal(sap.valid, true);
|
|
assert.equal(sap.method, "sap_models");
|
|
});
|
|
|
|
test("specialty validator accepts native Bedrock model discovery with a configured region", async () => {
|
|
const seenUrls: string[] = [];
|
|
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
seenUrls.push(target);
|
|
assert.equal((init.headers as Record<string, string>).Authorization, "Bearer bedrock-key");
|
|
|
|
if (
|
|
target === "https://bedrock.eu-west-2.amazonaws.com/foundation-models?byOutputModality=TEXT"
|
|
) {
|
|
return new Response(
|
|
JSON.stringify({
|
|
modelSummaries: [
|
|
{
|
|
modelId: "anthropic.claude-sonnet-4-6",
|
|
modelName: "Claude Sonnet 4.6",
|
|
providerName: "Anthropic",
|
|
inputModalities: ["TEXT", "IMAGE"],
|
|
outputModalities: ["TEXT"],
|
|
responseStreamingSupported: true,
|
|
},
|
|
],
|
|
}),
|
|
{ status: 200 }
|
|
);
|
|
}
|
|
|
|
if (
|
|
target ===
|
|
"https://bedrock.eu-west-2.amazonaws.com/inference-profiles?maxResults=100&typeEquals=SYSTEM_DEFINED"
|
|
) {
|
|
return new Response(
|
|
JSON.stringify({
|
|
inferenceProfileSummaries: [
|
|
{
|
|
inferenceProfileId: "eu.anthropic.claude-sonnet-4-6",
|
|
inferenceProfileName: "EU Claude Sonnet 4.6",
|
|
models: [
|
|
{
|
|
modelArn:
|
|
"arn:aws:bedrock:eu-west-2::foundation-model/anthropic.claude-sonnet-4-6",
|
|
},
|
|
],
|
|
},
|
|
],
|
|
}),
|
|
{ status: 200 }
|
|
);
|
|
}
|
|
|
|
throw new Error("unexpected fetch: " + target);
|
|
};
|
|
|
|
const bedrock = await validateProviderApiKey({
|
|
provider: "bedrock",
|
|
apiKey: "bedrock-key",
|
|
providerSpecificData: { region: "eu-west-2" },
|
|
});
|
|
|
|
assert.equal(bedrock.valid, true);
|
|
assert.equal(bedrock.method, "bedrock_native_models");
|
|
assert.deepEqual(seenUrls, [
|
|
"https://bedrock.eu-west-2.amazonaws.com/foundation-models?byOutputModality=TEXT",
|
|
"https://bedrock.eu-west-2.amazonaws.com/inference-profiles?maxResults=100&typeEquals=SYSTEM_DEFINED",
|
|
]);
|
|
});
|
|
|
|
test("specialty validator rejects invalid native Bedrock credentials", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
|
|
if (
|
|
target === "https://bedrock.eu-west-2.amazonaws.com/foundation-models?byOutputModality=TEXT"
|
|
) {
|
|
assert.equal((init.headers as Record<string, string>).Authorization, "Bearer bedrock-key");
|
|
return new Response(JSON.stringify({ message: "forbidden" }), { status: 403 });
|
|
}
|
|
|
|
throw new Error("unexpected fetch: " + target);
|
|
};
|
|
|
|
const bedrock = await validateProviderApiKey({
|
|
provider: "bedrock",
|
|
apiKey: "bedrock-key",
|
|
providerSpecificData: { region: "eu-west-2" },
|
|
});
|
|
|
|
assert.equal(bedrock.error, "Invalid API key");
|
|
});
|
|
|
|
test("specialty validators reject invalid watsonx, OCI and SAP credentials", async () => {
|
|
globalThis.fetch = async (url) => {
|
|
const target = String(url);
|
|
|
|
if (target === "https://ca-tor.ml.cloud.ibm.com/ml/gateway/v1/models") {
|
|
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
|
|
}
|
|
|
|
if (
|
|
target === "https://inference.generativeai.us-chicago-1.oci.oraclecloud.com/openai/v1/models"
|
|
) {
|
|
return new Response(JSON.stringify({ error: "forbidden" }), { status: 403 });
|
|
}
|
|
|
|
if (target === "https://sap.example.com/v2/lm/scenarios/foundation-models/models") {
|
|
return new Response(JSON.stringify({ error: "forbidden" }), { status: 403 });
|
|
}
|
|
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const watsonx = await validateProviderApiKey({
|
|
provider: "watsonx",
|
|
apiKey: "watsonx-key",
|
|
providerSpecificData: { baseUrl: "https://ca-tor.ml.cloud.ibm.com" },
|
|
});
|
|
const oci = await validateProviderApiKey({
|
|
provider: "oci",
|
|
apiKey: "oci-key",
|
|
providerSpecificData: {
|
|
baseUrl: "https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
|
|
},
|
|
});
|
|
const sap = await validateProviderApiKey({
|
|
provider: "sap",
|
|
apiKey: "sap-key",
|
|
providerSpecificData: {
|
|
baseUrl: "https://sap.example.com/v2/lm/deployments/demo-deployment",
|
|
},
|
|
});
|
|
|
|
assert.equal(watsonx.error, "Invalid API key");
|
|
assert.equal(oci.error, "Invalid API key");
|
|
assert.equal(sap.error, "Invalid API key");
|
|
});
|
|
|
|
test("specialty validator accepts Modal OpenAI-compatible deployments", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
|
|
if (target === "https://alice--demo.modal.run/v1/models") {
|
|
const headers = init.headers as Record<string, string>;
|
|
assert.equal(headers.Authorization, "Bearer modal-key");
|
|
return new Response(JSON.stringify({ data: [] }), { status: 200 });
|
|
}
|
|
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const modal = await validateProviderApiKey({
|
|
provider: "modal",
|
|
apiKey: "modal-key",
|
|
providerSpecificData: {
|
|
baseUrl: "https://alice--demo.modal.run/v1",
|
|
},
|
|
});
|
|
|
|
assert.equal(modal.valid, true);
|
|
});
|
|
|
|
test("specialty validator rejects invalid Modal credentials", async () => {
|
|
globalThis.fetch = async (url) => {
|
|
const target = String(url);
|
|
|
|
if (target === "https://alice--demo.modal.run/v1/models") {
|
|
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
|
|
}
|
|
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const modal = await validateProviderApiKey({
|
|
provider: "modal",
|
|
apiKey: "modal-key",
|
|
providerSpecificData: {
|
|
baseUrl: "https://alice--demo.modal.run/v1",
|
|
},
|
|
});
|
|
|
|
assert.equal(modal.error, "Invalid API key");
|
|
});
|
|
|
|
test("specialty validator accepts Poe credentials on the current balance endpoint", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
|
|
if (target === "https://api.poe.com/usage/current_balance") {
|
|
const headers = init.headers as Record<string, string>;
|
|
assert.equal(headers.Authorization, "Bearer poe-key");
|
|
return new Response(JSON.stringify({ current_point_balance: 123456 }), { status: 200 });
|
|
}
|
|
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const poe = await validateProviderApiKey({
|
|
provider: "poe",
|
|
apiKey: "poe-key",
|
|
});
|
|
|
|
assert.equal(poe.valid, true);
|
|
assert.equal(poe.method, "poe_current_balance");
|
|
});
|
|
|
|
test("specialty validator accepts Nous Research credentials on chat completions", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
|
|
if (target === "https://inference-api.nousresearch.com/v1/chat/completions") {
|
|
const headers = init.headers as Record<string, string>;
|
|
const body = JSON.parse(String(init.body));
|
|
assert.equal(headers.Authorization, "Bearer nous-key");
|
|
assert.equal(body.model, "Hermes-4-70B");
|
|
return new Response(
|
|
JSON.stringify({
|
|
id: "chatcmpl-nous",
|
|
choices: [{ message: { role: "assistant", content: "ok" } }],
|
|
}),
|
|
{ status: 200 }
|
|
);
|
|
}
|
|
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const nous = await validateProviderApiKey({
|
|
provider: "nous-research",
|
|
apiKey: "nous-key",
|
|
});
|
|
|
|
assert.equal(nous.valid, true);
|
|
assert.equal(nous.method, "nous_chat_completions");
|
|
});
|
|
|
|
test("BytePlus key validation reaches the Ark endpoint instead of 'not supported' (#3877)", async () => {
|
|
// #3877: byteplus was in APIKEY_PROVIDERS but never registered in the routing
|
|
// registry, so validation returned {unsupported:true} → UI showed "invalid" for any
|
|
// key. With the registry entry, a valid ark-... key probes the Ark /models endpoint
|
|
// with Bearer auth and validates.
|
|
let probedModelsUrl: string | null = null;
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
if (target === "https://ark.ap-southeast.bytepluses.com/api/v3/models") {
|
|
probedModelsUrl = target;
|
|
const headers = toPlainHeaders(init.headers);
|
|
assert.equal(headers.Authorization, "Bearer ark-test-key");
|
|
return new Response(JSON.stringify({ data: [{ id: "kimi-k2-thinking" }] }), { status: 200 });
|
|
}
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "byteplus",
|
|
apiKey: "ark-test-key",
|
|
});
|
|
|
|
assert.equal(result.unsupported, undefined, "byteplus must not be 'validation not supported'");
|
|
assert.equal(result.valid, true);
|
|
assert.equal(probedModelsUrl, "https://ark.ap-southeast.bytepluses.com/api/v3/models");
|
|
});
|
|
|
|
test("specialty validator rejects invalid Nous Research credentials", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
|
|
if (target === "https://inference-api.nousresearch.com/v1/chat/completions") {
|
|
const headers = init.headers as Record<string, string>;
|
|
assert.equal(headers.Authorization, "Bearer nous-bad");
|
|
return new Response(JSON.stringify({ message: "invalid" }), { status: 401 });
|
|
}
|
|
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const nous = await validateProviderApiKey({
|
|
provider: "nous-research",
|
|
apiKey: "nous-bad",
|
|
});
|
|
|
|
assert.equal(nous.error, "Invalid API key");
|
|
});
|
|
|
|
test("specialty validator accepts Nous Research key when probe model is rejected (400)", async () => {
|
|
// #3881: a valid key whose probe model is rejected (model-not-found / bad request)
|
|
// must still validate — the 4xx proves auth was accepted, only the request shape
|
|
// was wrong. Mirrors the longcat/nvidia validators.
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
|
|
if (target === "https://inference-api.nousresearch.com/v1/chat/completions") {
|
|
const headers = init.headers as Record<string, string>;
|
|
assert.equal(headers.Authorization, "Bearer nous-key");
|
|
return new Response(
|
|
JSON.stringify({ error: { message: "model not found", type: "invalid_request_error" } }),
|
|
{ status: 400 }
|
|
);
|
|
}
|
|
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const nous = await validateProviderApiKey({
|
|
provider: "nous-research",
|
|
apiKey: "nous-key",
|
|
});
|
|
|
|
assert.equal(nous.valid, true);
|
|
assert.equal(nous.method, "nous_chat_completions");
|
|
});
|
|
|
|
test("specialty validator rejects invalid Poe credentials", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
|
|
if (target === "https://api.poe.com/usage/current_balance") {
|
|
const headers = init.headers as Record<string, string>;
|
|
assert.equal(headers.Authorization, "Bearer poe-bad");
|
|
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
|
|
}
|
|
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const poe = await validateProviderApiKey({
|
|
provider: "poe",
|
|
apiKey: "poe-bad",
|
|
});
|
|
|
|
assert.equal(poe.error, "Invalid API key");
|
|
});
|
|
|
|
test("specialty validator accepts Clarifai credentials through the OpenAI-compatible models probe", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
|
|
if (target === "https://api.clarifai.com/v2/ext/openai/v1/models") {
|
|
const headers = init.headers as Record<string, string>;
|
|
assert.equal(headers.Authorization, "Key clarifai-pat");
|
|
return new Response(
|
|
JSON.stringify({ data: [{ id: "openai/chat-completion/models/gpt-oss-120b" }] }),
|
|
{ status: 200 }
|
|
);
|
|
}
|
|
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const clarifai = await validateProviderApiKey({
|
|
provider: "clarifai",
|
|
apiKey: "clarifai-pat",
|
|
});
|
|
|
|
assert.equal(clarifai.valid, true);
|
|
assert.equal(clarifai.method, "clarifai_models");
|
|
});
|
|
|
|
test("specialty validator rejects invalid Clarifai credentials", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
|
|
if (target === "https://api.clarifai.com/v2/ext/openai/v1/models") {
|
|
const headers = init.headers as Record<string, string>;
|
|
assert.equal(headers.Authorization, "Key clarifai-bad");
|
|
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
|
|
}
|
|
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const clarifai = await validateProviderApiKey({
|
|
provider: "clarifai",
|
|
apiKey: "clarifai-bad",
|
|
});
|
|
|
|
assert.equal(clarifai.error, "Invalid API key");
|
|
});
|
|
|
|
test("specialty validator accepts Reka credentials through the models probe with dual auth headers", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
|
|
if (target === "https://api.reka.ai/v1/models") {
|
|
const headers = init.headers as Record<string, string>;
|
|
assert.equal(headers.Authorization, "Bearer reka-key");
|
|
assert.equal(headers["X-Api-Key"], "reka-key");
|
|
return new Response(JSON.stringify([{ id: "reka-core" }]), { status: 200 });
|
|
}
|
|
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const reka = await validateProviderApiKey({
|
|
provider: "reka",
|
|
apiKey: "reka-key",
|
|
});
|
|
|
|
assert.equal(reka.valid, true);
|
|
assert.equal(reka.method, "reka_models");
|
|
});
|
|
|
|
test("specialty validator rejects invalid Reka credentials", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
|
|
if (target === "https://api.reka.ai/v1/models") {
|
|
const headers = init.headers as Record<string, string>;
|
|
assert.equal(headers.Authorization, "Bearer reka-bad");
|
|
assert.equal(headers["X-Api-Key"], "reka-bad");
|
|
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
|
|
}
|
|
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const reka = await validateProviderApiKey({
|
|
provider: "reka",
|
|
apiKey: "reka-bad",
|
|
});
|
|
|
|
assert.equal(reka.error, "Invalid API key");
|
|
});
|
|
|
|
test("specialty validator accepts NLP Cloud credentials on the chatbot endpoint", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
|
|
if (target === "https://api.nlpcloud.io/v1/gpu/chatdolphin/chatbot") {
|
|
const headers = init.headers as Record<string, string>;
|
|
const body = JSON.parse(String(init.body));
|
|
assert.equal(headers.Authorization, "Token nlpc-key");
|
|
assert.equal(body.input, "test");
|
|
return new Response(JSON.stringify({ response: "ok" }), { status: 200 });
|
|
}
|
|
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const nlpCloud = await validateProviderApiKey({
|
|
provider: "nlpcloud",
|
|
apiKey: "nlpc-key",
|
|
});
|
|
|
|
assert.equal(nlpCloud.valid, true);
|
|
assert.equal(nlpCloud.method, "nlpcloud_chatbot");
|
|
});
|
|
|
|
test("specialty validator rejects invalid NLP Cloud credentials", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
|
|
if (target === "https://api.nlpcloud.io/v1/gpu/chatdolphin/chatbot") {
|
|
const headers = init.headers as Record<string, string>;
|
|
assert.equal(headers.Authorization, "Token nlpc-bad");
|
|
return new Response(JSON.stringify({ detail: "forbidden" }), { status: 403 });
|
|
}
|
|
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const nlpCloud = await validateProviderApiKey({
|
|
provider: "nlpcloud",
|
|
apiKey: "nlpc-bad",
|
|
});
|
|
|
|
assert.equal(nlpCloud.error, "Invalid API key");
|
|
});
|
|
|
|
test("specialty validator accepts Runway credentials on the organization endpoint", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
|
|
if (target === "https://api.dev.runwayml.com/v1/organization") {
|
|
const headers = init.headers as Record<string, string>;
|
|
assert.equal(headers.Authorization, "Bearer runway-key");
|
|
assert.equal(headers["X-Runway-Version"], "2024-11-06");
|
|
return new Response(JSON.stringify({ id: "org_demo" }), { status: 200 });
|
|
}
|
|
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const runway = await validateProviderApiKey({
|
|
provider: "runwayml",
|
|
apiKey: "runway-key",
|
|
});
|
|
|
|
assert.equal(runway.valid, true);
|
|
assert.equal(runway.method, "runway_organization");
|
|
});
|
|
|
|
test("specialty validator rejects invalid Runway credentials", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
|
|
if (target === "https://api.dev.runwayml.com/v1/organization") {
|
|
const headers = init.headers as Record<string, string>;
|
|
assert.equal(headers.Authorization, "Bearer runway-bad");
|
|
assert.equal(headers["X-Runway-Version"], "2024-11-06");
|
|
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
|
|
}
|
|
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const runway = await validateProviderApiKey({
|
|
provider: "runwayml",
|
|
apiKey: "runway-bad",
|
|
});
|
|
|
|
assert.equal(runway.error, "Invalid API key");
|
|
});
|
|
|
|
test("validateCommandCodeProvider sends Command Code probe URL, headers, and wrapper body", async () => {
|
|
const calls: any[] = [];
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
calls.push({
|
|
url: String(url),
|
|
method: init.method,
|
|
headers: toPlainHeaders(init.headers),
|
|
body: JSON.parse(String(init.body)),
|
|
});
|
|
return new Response("", { status: 400 });
|
|
};
|
|
|
|
const result = await validateCommandCodeProvider({
|
|
apiKey: "cc_test_key",
|
|
providerSpecificData: { validationModelId: "gpt-5.4-mini" },
|
|
});
|
|
|
|
assert.deepEqual(result, { valid: true, error: null });
|
|
assert.equal(calls.length, 1);
|
|
assert.equal(calls[0].url, "https://api.commandcode.ai/alpha/generate");
|
|
assert.equal(calls[0].method, "POST");
|
|
assert.equal(calls[0].headers.Authorization, "Bearer cc_test_key");
|
|
assert.equal(calls[0].headers["Content-Type"], "application/json");
|
|
assert.equal(calls[0].headers["x-command-code-version"], COMMAND_CODE_VERSION);
|
|
assert.equal(calls[0].headers["x-cli-environment"], "external");
|
|
assert.equal(calls[0].headers["x-project-slug"], "pi-cc");
|
|
assert.equal(calls[0].headers["x-taste-learning"], "false");
|
|
assert.equal(calls[0].headers["x-co-flag"], "false");
|
|
assert.equal(typeof calls[0].headers["x-session-id"], "string");
|
|
assert.equal(calls[0].body.config.environment, "external");
|
|
assert.equal(calls[0].body.permissionMode, "standard");
|
|
assert.equal(calls[0].body.skills, "");
|
|
assert.equal(calls[0].body.params.model, "gpt-5.4-mini");
|
|
assert.equal(calls[0].body.params.stream, true);
|
|
assert.equal(calls[0].body.params.max_tokens, 1);
|
|
});
|
|
|
|
for (const status of [400, 422, 429]) {
|
|
test(`validateCommandCodeProvider accepts ${status} as direct validator auth success`, async () => {
|
|
globalThis.fetch = async () => new Response("", { status });
|
|
assert.deepEqual(await validateCommandCodeProvider({ apiKey: "cc_test_key" }), {
|
|
valid: true,
|
|
error: null,
|
|
});
|
|
});
|
|
}
|
|
|
|
test("validateCommandCodeProvider rejects auth failures and provider outages", async () => {
|
|
globalThis.fetch = async () => new Response("unauthorized", { status: 401 });
|
|
assert.deepEqual(await validateCommandCodeProvider({ apiKey: "bad" }), {
|
|
valid: false,
|
|
error: "Invalid API key",
|
|
});
|
|
|
|
globalThis.fetch = async () => new Response("server down", { status: 500 });
|
|
assert.deepEqual(await validateCommandCodeProvider({ apiKey: "cc_test_key" }), {
|
|
valid: false,
|
|
error: "Provider unavailable (500)",
|
|
});
|
|
});
|
|
|
|
// ─── claude-web validator ────────────────────────────────────────────────────
|
|
|
|
const { __setTlsFetchOverrideForTesting: __setClaudeTlsFetchOverride } =
|
|
await import("../../open-sse/services/claudeTlsClient.ts");
|
|
|
|
function makeClaudeTlsResponse(
|
|
status: number,
|
|
body: string,
|
|
headers: Record<string, string> = {}
|
|
): any {
|
|
const h = new Headers();
|
|
for (const [k, v] of Object.entries(headers)) h.set(k, v);
|
|
return { status, ok: status >= 200 && status < 300, headers: h, text: body, body: null };
|
|
}
|
|
|
|
test("claude-web validator: 200 from /api/organizations → valid", async () => {
|
|
let captured: { url: string; opts: any } | null = null;
|
|
__setClaudeTlsFetchOverride(async (url, opts) => {
|
|
captured = { url, opts };
|
|
return makeClaudeTlsResponse(200, JSON.stringify({ orgs: [] }));
|
|
});
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "claude-web",
|
|
apiKey: "sessionKey=sk-ant-sid02-test-session-key",
|
|
});
|
|
|
|
assert.equal(result.valid, true);
|
|
assert.equal(result.error, null);
|
|
assert.equal(captured?.url, "https://claude.ai/api/organizations");
|
|
assert.match(
|
|
(captured?.opts.headers as Record<string, string>).Cookie || "",
|
|
/sessionKey=sk-ant-sid02-test-session-key/
|
|
);
|
|
__setClaudeTlsFetchOverride(null);
|
|
});
|
|
|
|
test("claude-web validator: full cookie blob passes through verbatim", async () => {
|
|
let capturedCookie = "";
|
|
__setClaudeTlsFetchOverride(async (_url, opts) => {
|
|
capturedCookie = (opts.headers as Record<string, string>).Cookie || "";
|
|
return makeClaudeTlsResponse(200, JSON.stringify({ orgs: [] }));
|
|
});
|
|
|
|
const blob =
|
|
"__cf_bm=abc123; sessionKey=sk-ant-sid02-test; intercom-device-id-lupk8zyo=xyz; __stripe_mid=stripe123";
|
|
await validateProviderApiKey({ provider: "claude-web", apiKey: blob });
|
|
assert.equal(capturedCookie, blob);
|
|
__setClaudeTlsFetchOverride(null);
|
|
});
|
|
|
|
test("claude-web validator: 401 → invalid session cookie", async () => {
|
|
__setClaudeTlsFetchOverride(async () =>
|
|
makeClaudeTlsResponse(401, JSON.stringify({ error: "unauthorized" }))
|
|
);
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "claude-web",
|
|
apiKey: "sessionKey=expired-key",
|
|
});
|
|
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /Invalid or expired session cookie/i);
|
|
__setClaudeTlsFetchOverride(null);
|
|
});
|
|
|
|
test("claude-web validator: 429 → valid (rate limited means auth passed)", async () => {
|
|
__setClaudeTlsFetchOverride(async () =>
|
|
makeClaudeTlsResponse(429, JSON.stringify({ error: "rate limited" }))
|
|
);
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "claude-web",
|
|
apiKey: "sessionKey=sk-ant-sid02-good-key",
|
|
});
|
|
|
|
assert.equal(result.valid, true);
|
|
__setClaudeTlsFetchOverride(null);
|
|
});
|
|
|
|
test("claude-web validator: 500 → Claude.ai unavailable", async () => {
|
|
__setClaudeTlsFetchOverride(async () => makeClaudeTlsResponse(500, "internal server error"));
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "claude-web",
|
|
apiKey: "sessionKey=sk-ant-sid02-any-key",
|
|
});
|
|
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /Claude\.ai unavailable \(500\)/i);
|
|
__setClaudeTlsFetchOverride(null);
|
|
});
|
|
|
|
test("claude-web validator: TLS client unavailable → clear error", async () => {
|
|
const { TlsClientUnavailableError } = await import("../../open-sse/services/claudeTlsClient.ts");
|
|
__setClaudeTlsFetchOverride(async () => {
|
|
throw new TlsClientUnavailableError("tls-client-node not installed");
|
|
});
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "claude-web",
|
|
apiKey: "sessionKey=sk-ant-sid02-any-key",
|
|
});
|
|
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /tls-client-node not installed/i);
|
|
__setClaudeTlsFetchOverride(null);
|
|
});
|
|
|
|
test("claude-web validator: bare sessionKey value gets prefixed", async () => {
|
|
let capturedCookie = "";
|
|
__setClaudeTlsFetchOverride(async (_url, opts) => {
|
|
capturedCookie = (opts.headers as Record<string, string>).Cookie || "";
|
|
return makeClaudeTlsResponse(200, JSON.stringify({ orgs: [] }));
|
|
});
|
|
|
|
await validateProviderApiKey({
|
|
provider: "claude-web",
|
|
apiKey: "sk-ant-sid02-bare-value",
|
|
});
|
|
assert.equal(capturedCookie, "sessionKey=sk-ant-sid02-bare-value");
|
|
__setClaudeTlsFetchOverride(null);
|
|
});
|
|
|
|
// ─── gemini-web validator ────────────────────────────────────────────────────
|
|
|
|
test("gemini-web validator: 200 from gemini.google.com → valid", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
const headers = init.headers || {};
|
|
if (target.includes("gemini.google.com/app")) {
|
|
assert.match((headers as Record<string, string>).Cookie || "", /__Secure-1PSID=eyJPSID/);
|
|
return new Response("ok", { status: 200 });
|
|
}
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "gemini-web",
|
|
apiKey: "__Secure-1PSID=eyJPSID",
|
|
});
|
|
|
|
assert.equal(result.valid, true);
|
|
assert.equal(result.error, null);
|
|
});
|
|
|
|
test("gemini-web validator: bare value gets __Secure-1PSID prefix", async () => {
|
|
let capturedCookie = "";
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
if (String(url).includes("gemini.google.com")) {
|
|
capturedCookie = ((init.headers as Record<string, string>) || {}).Cookie || "";
|
|
return new Response("ok", { status: 200 });
|
|
}
|
|
throw new Error(`unexpected fetch: ${String(url)}`);
|
|
};
|
|
|
|
await validateProviderApiKey({ provider: "gemini-web", apiKey: "eyJbarevalue" });
|
|
assert.equal(capturedCookie, "__Secure-1PSID=eyJbarevalue");
|
|
});
|
|
|
|
test("gemini-web validator: 401 → invalid cookie", async () => {
|
|
globalThis.fetch = async () => new Response("unauthorized", { status: 401 });
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "gemini-web",
|
|
apiKey: "__Secure-1PSID=expired",
|
|
});
|
|
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /Invalid or expired __Secure-1PSID/i);
|
|
});
|
|
|
|
test("gemini-web validator: 500 → unavailable", async () => {
|
|
globalThis.fetch = async () => new Response("down", { status: 500 });
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "gemini-web",
|
|
apiKey: "__Secure-1PSID=eyJany",
|
|
});
|
|
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /Gemini validation failed \(500\)/i);
|
|
});
|
|
|
|
// ─── copilot-web validator ───────────────────────────────────────────────────
|
|
|
|
test("copilot-web validator: valid access_token → 200", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
const target = String(url);
|
|
if (target.includes("copilot.microsoft.com/c/api/conversations")) {
|
|
assert.match(
|
|
((init.headers as Record<string, string>) || {}).Authorization || "",
|
|
/Bearer eyJhbGci/
|
|
);
|
|
return new Response(JSON.stringify({ conversations: [] }), { status: 200 });
|
|
}
|
|
throw new Error(`unexpected fetch: ${target}`);
|
|
};
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "copilot-web",
|
|
apiKey: "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.test",
|
|
});
|
|
|
|
assert.equal(result.valid, true);
|
|
assert.equal(result.error, null);
|
|
});
|
|
|
|
test("copilot-web validator: cookie with access_token= is extracted", async () => {
|
|
let capturedAuth = "";
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
if (String(url).includes("copilot.microsoft.com")) {
|
|
capturedAuth = ((init.headers as Record<string, string>) || {}).Authorization || "";
|
|
return new Response(JSON.stringify({}), { status: 200 });
|
|
}
|
|
throw new Error(`unexpected fetch: ${String(url)}`);
|
|
};
|
|
|
|
await validateProviderApiKey({
|
|
provider: "copilot-web",
|
|
apiKey: "access_token=eyJhbGciOiJIUzI1NiJ9.payload.sig; other_cookie=foo",
|
|
});
|
|
assert.equal(capturedAuth, "Bearer eyJhbGciOiJIUzI1NiJ9.payload.sig");
|
|
});
|
|
|
|
test("copilot-web validator: 401 → invalid token", async () => {
|
|
globalThis.fetch = async () => new Response("unauthorized", { status: 401 });
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "copilot-web",
|
|
apiKey: "bad-token",
|
|
});
|
|
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /Invalid or expired access_token/i);
|
|
});
|
|
|
|
test("copilot-web validator: 500 → unavailable", async () => {
|
|
globalThis.fetch = async () => new Response("down", { status: 500 });
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "copilot-web",
|
|
apiKey: "any-token",
|
|
});
|
|
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /Copilot unavailable \(500\)/i);
|
|
});
|
|
|
|
test("copilot-web validator: empty input → paste prompt", async () => {
|
|
globalThis.fetch = async () => {
|
|
throw new Error("should not fetch");
|
|
};
|
|
|
|
const result = await validateProviderApiKey({ provider: "copilot-web", apiKey: "" });
|
|
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /Paste your access_token/i);
|
|
});
|
|
|
|
// ─── t3-web validator ────────────────────────────────────────────────────────
|
|
|
|
test("t3-web validator: valid cookies → valid", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
if (String(url).includes("t3.chat")) {
|
|
return new Response("ok", { status: 200 });
|
|
}
|
|
throw new Error(`unexpected fetch: ${String(url)}`);
|
|
};
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "t3-web",
|
|
apiKey: "cookies=__session=abc123; convexSessionId=def456",
|
|
});
|
|
|
|
assert.equal(result.valid, true);
|
|
assert.equal(result.error, null);
|
|
});
|
|
|
|
test("t3-web validator: 500 → unavailable", async () => {
|
|
globalThis.fetch = async () => new Response("down", { status: 500 });
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "t3-web",
|
|
apiKey: "cookies=__session=abc",
|
|
});
|
|
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /t3\.chat unavailable \(500\)/i);
|
|
});
|
|
|
|
test("t3-web validator: valid cookies → passes through", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
if (String(url).includes("t3.chat")) {
|
|
return new Response("ok", { status: 200 });
|
|
}
|
|
throw new Error(`unexpected fetch: ${String(url)}`);
|
|
};
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "t3-web",
|
|
apiKey: "__session=abc123; convex-session-id=def456",
|
|
});
|
|
|
|
assert.equal(result.valid, true);
|
|
assert.equal(result.error, null);
|
|
});
|
|
|
|
test("llama-cpp is classified as a self-hosted chat provider", async () => {
|
|
const { isSelfHostedChatProvider, isLocalProvider, providerAllowsOptionalApiKey } =
|
|
await import("../../src/shared/constants/providers.ts");
|
|
|
|
assert.equal(isSelfHostedChatProvider("llama-cpp"), true);
|
|
assert.equal(isLocalProvider("llama-cpp"), true);
|
|
assert.equal(providerAllowsOptionalApiKey("llama-cpp"), true);
|
|
});
|
|
|
|
// ─── Gitlawb Opengateway specialty validators ──────────────────────────────
|
|
|
|
test("gitlawb validator: accepts valid API key via chat/completions probe", async () => {
|
|
const calls: any[] = [];
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
calls.push({ url: String(url), headers: init.headers || {}, body: init.body });
|
|
assert.equal(String(url), "https://opengateway.gitlawb.com/v1/xiaomi-mimo/chat/completions");
|
|
assert.equal((init.headers as Record<string, string>).Authorization, "Bearer glb-valid-key");
|
|
const body = JSON.parse(String(init.body));
|
|
assert.equal(body.model, "mimo-v2.5-pro");
|
|
assert.equal(body.messages[0].content, "test");
|
|
assert.equal(body.max_tokens, 1);
|
|
return new Response(JSON.stringify({ choices: [{ message: { content: "ok" } }] }), {
|
|
status: 200,
|
|
});
|
|
};
|
|
|
|
const result = await validateProviderApiKey({ provider: "gitlawb", apiKey: "glb-valid-key" });
|
|
assert.equal(result.valid, true);
|
|
assert.equal(calls.length, 1);
|
|
});
|
|
|
|
test("gitlawb validator: 400/422/429 treated as auth success", async () => {
|
|
for (const status of [400, 422, 429]) {
|
|
globalThis.fetch = async (url) => {
|
|
assert.equal(String(url), "https://opengateway.gitlawb.com/v1/xiaomi-mimo/chat/completions");
|
|
return new Response(JSON.stringify({ error: "bad request" }), { status });
|
|
};
|
|
const result = await validateProviderApiKey({ provider: "gitlawb", apiKey: "glb-key" });
|
|
assert.equal(result.valid, true, `status ${status} should pass auth`);
|
|
assert.equal(result.error, null, `status ${status} should not return error`);
|
|
}
|
|
});
|
|
|
|
test("gitlawb validator: rejects invalid API key (401)", async () => {
|
|
globalThis.fetch = async (url) => {
|
|
assert.equal(String(url), "https://opengateway.gitlawb.com/v1/xiaomi-mimo/chat/completions");
|
|
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
|
|
};
|
|
|
|
const result = await validateProviderApiKey({ provider: "gitlawb", apiKey: "glb-bad-key" });
|
|
assert.equal(result.valid, false);
|
|
assert.equal(result.error, "Invalid API key");
|
|
});
|
|
|
|
test("gitlawb validator: rejects invalid API key (403)", async () => {
|
|
globalThis.fetch = async (url) => {
|
|
assert.equal(String(url), "https://opengateway.gitlawb.com/v1/xiaomi-mimo/chat/completions");
|
|
return new Response(JSON.stringify({ error: "forbidden" }), { status: 403 });
|
|
};
|
|
|
|
const result = await validateProviderApiKey({ provider: "gitlawb", apiKey: "glb-bad-key" });
|
|
assert.equal(result.valid, false);
|
|
assert.equal(result.error, "Invalid API key");
|
|
});
|
|
|
|
test("gitlawb validator: surfaces network failures", async () => {
|
|
globalThis.fetch = async () => {
|
|
throw new Error("gitlawb opengateway offline");
|
|
};
|
|
|
|
const result = await validateProviderApiKey({ provider: "gitlawb", apiKey: "glb-key" });
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /gitlawb opengateway offline/i);
|
|
});
|
|
|
|
test("gitlawb validator: accepts custom baseUrl override", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
assert.equal(String(url), "https://custom-gateway.example.com/v1/xiaomi-mimo/chat/completions");
|
|
assert.equal((init.headers as Record<string, string>).Authorization, "Bearer glb-key");
|
|
return new Response(JSON.stringify({ choices: [{ message: { content: "ok" } }] }), {
|
|
status: 200,
|
|
});
|
|
};
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "gitlawb",
|
|
apiKey: "glb-key",
|
|
providerSpecificData: {
|
|
baseUrl: "https://custom-gateway.example.com/v1/xiaomi-mimo",
|
|
},
|
|
});
|
|
assert.equal(result.valid, true);
|
|
});
|
|
|
|
// ─── Gitlawb-GMI (GMI Cloud) ─────────────────────────────────────────────
|
|
|
|
test("gitlawb-gmi validator: accepts valid API key via chat/completions probe", async () => {
|
|
const calls: any[] = [];
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
calls.push({ url: String(url), headers: init.headers || {} });
|
|
assert.equal(String(url), "https://opengateway.gitlawb.com/v1/gmi-cloud/chat/completions");
|
|
assert.equal(
|
|
(init.headers as Record<string, string>).Authorization,
|
|
"Bearer glb-gmi-valid-key"
|
|
);
|
|
const body = JSON.parse(String(init.body));
|
|
assert.equal(body.model, "XiaomiMiMo/MiMo-V2.5-Pro");
|
|
assert.equal(body.messages[0].content, "test");
|
|
assert.equal(body.max_tokens, 1);
|
|
return new Response(JSON.stringify({ choices: [{ message: { content: "ok" } }] }), {
|
|
status: 200,
|
|
});
|
|
};
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "gitlawb-gmi",
|
|
apiKey: "glb-gmi-valid-key",
|
|
});
|
|
assert.equal(result.valid, true);
|
|
assert.equal(calls.length, 1);
|
|
});
|
|
|
|
test("gitlawb-gmi validator: accepts 400/422/429 as auth success", async () => {
|
|
for (const status of [400, 422, 429]) {
|
|
globalThis.fetch = async (url) => {
|
|
assert.equal(String(url), "https://opengateway.gitlawb.com/v1/gmi-cloud/chat/completions");
|
|
return new Response(JSON.stringify({ error: "bad request" }), { status });
|
|
};
|
|
const result = await validateProviderApiKey({
|
|
provider: "gitlawb-gmi",
|
|
apiKey: "glb-gmi-key",
|
|
});
|
|
assert.equal(result.valid, true, `status ${status} should pass auth`);
|
|
}
|
|
});
|
|
|
|
test("gitlawb-gmi validator: rejects invalid API key (401)", async () => {
|
|
globalThis.fetch = async (url) => {
|
|
assert.equal(String(url), "https://opengateway.gitlawb.com/v1/gmi-cloud/chat/completions");
|
|
return new Response(JSON.stringify({ error: "unauthorized" }), { status: 401 });
|
|
};
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "gitlawb-gmi",
|
|
apiKey: "glb-gmi-bad-key",
|
|
});
|
|
assert.equal(result.valid, false);
|
|
assert.equal(result.error, "Invalid API key");
|
|
});
|
|
|
|
test("gitlawb-gmi validator: rejects invalid API key (403)", async () => {
|
|
globalThis.fetch = async (url) => {
|
|
assert.equal(String(url), "https://opengateway.gitlawb.com/v1/gmi-cloud/chat/completions");
|
|
return new Response(JSON.stringify({ error: "forbidden" }), { status: 403 });
|
|
};
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "gitlawb-gmi",
|
|
apiKey: "glb-gmi-bad-key",
|
|
});
|
|
assert.equal(result.valid, false);
|
|
assert.equal(result.error, "Invalid API key");
|
|
});
|
|
|
|
test("gitlawb-gmi validator: surfaces network failures", async () => {
|
|
globalThis.fetch = async () => {
|
|
throw new Error("gitlawb-gmi opengateway offline");
|
|
};
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "gitlawb-gmi",
|
|
apiKey: "glb-gmi-key",
|
|
});
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error || "", /gitlawb-gmi opengateway offline/i);
|
|
});
|
|
|
|
test("gitlawb-gmi validator: accepts custom baseUrl override", async () => {
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
assert.equal(String(url), "https://custom-gateway.example.com/v1/gmi-cloud/chat/completions");
|
|
assert.equal((init.headers as Record<string, string>).Authorization, "Bearer glb-gmi-key");
|
|
return new Response(JSON.stringify({ choices: [{ message: { content: "ok" } }] }), {
|
|
status: 200,
|
|
});
|
|
};
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "gitlawb-gmi",
|
|
apiKey: "glb-gmi-key",
|
|
providerSpecificData: {
|
|
baseUrl: "https://custom-gateway.example.com/v1/gmi-cloud",
|
|
},
|
|
});
|
|
assert.equal(result.valid, true);
|
|
});
|
|
|
|
// #3288 / #3758: qwen-web validation used to fall through to the generic
|
|
// OpenAI-compatible validator, which probed a non-existent `/api/v2/models` URL that
|
|
// answered with a 307 redirect — blocked by the outbound guard and mislabeled as an
|
|
// SSRF block. A specialty validator now probes the real session endpoint instead.
|
|
test("qwen-web validator probes /api/v2/user (not /api/v2/models) and returns valid on 200", async () => {
|
|
let probedUrl = "";
|
|
let sentHeaders: Record<string, string> = {};
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
probedUrl = String(url);
|
|
sentHeaders = toPlainHeaders(init.headers);
|
|
// Qwen's /api/v2/user returns a real user object for a valid session. Since
|
|
// #3958 the validator inspects the body for `user` (Qwen answers 200 even for
|
|
// invalid tokens), so a valid response must carry one.
|
|
return new Response(JSON.stringify({ user: { id: "u-1", name: "Tester" } }), {
|
|
status: 200,
|
|
headers: { "content-type": "application/json" },
|
|
});
|
|
};
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "qwen-web",
|
|
apiKey: "token=eyJqwen; cna=abc; ssxmod_itna=def",
|
|
});
|
|
|
|
assert.equal(probedUrl, "https://chat.qwen.ai/api/v2/user");
|
|
assert.ok(!probedUrl.includes("/api/v2/models"), "must not probe the bogus /api/v2/models URL");
|
|
assert.equal(sentHeaders.Authorization, "Bearer eyJqwen");
|
|
assert.equal(sentHeaders.source, "web");
|
|
assert.match(sentHeaders.Cookie, /token=eyJqwen/);
|
|
assert.equal(result.valid, true);
|
|
});
|
|
|
|
test("qwen-web validator reports an invalid session (401) without flagging a security block", async () => {
|
|
globalThis.fetch = async () =>
|
|
new Response(JSON.stringify({ error: "unauthorized" }), {
|
|
status: 401,
|
|
headers: { "content-type": "application/json" },
|
|
});
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "qwen-web",
|
|
apiKey: "token=stale; cna=abc; ssxmod_itna=def",
|
|
});
|
|
|
|
assert.equal(result.valid, false);
|
|
assert.equal((result as { securityBlocked?: boolean }).securityBlocked ?? false, false);
|
|
assert.match(result.error ?? "", /invalid or expired/i);
|
|
});
|
|
|
|
test("qwen-web validator surfaces the WAF/anti-bot HTML challenge as a re-login hint", async () => {
|
|
globalThis.fetch = async () =>
|
|
new Response("<html>aliyun_waf</html>", {
|
|
status: 200,
|
|
headers: { "content-type": "text/html" },
|
|
});
|
|
|
|
const result = await validateProviderApiKey({
|
|
provider: "qwen-web",
|
|
apiKey: "token=eyJqwen; cna=abc; ssxmod_itna=def",
|
|
});
|
|
|
|
assert.equal(result.valid, false);
|
|
assert.match(result.error ?? "", /WAF|Cookie header/i);
|
|
});
|
|
|
|
// #3288 / #3758: a blocked redirect (REDIRECT_BLOCKED) to a PUBLIC host is benign — the
|
|
// redirect was never followed, so it must NOT be mislabeled as an SSRF security block.
|
|
// Only a redirect whose target is a private/internal host is a genuine security event.
|
|
test("isSecurityBlockError: public-host redirect block is NOT a security block", () => {
|
|
const publicRedirect = new SafeOutboundFetchError("Redirect blocked", {
|
|
code: "REDIRECT_BLOCKED",
|
|
url: "https://chat.qwen.ai/api/v2/models",
|
|
method: "GET",
|
|
attempts: 1,
|
|
status: 307,
|
|
location: "https://chat.qwen.ai/login",
|
|
isRetryable: false,
|
|
});
|
|
assert.equal(isSecurityBlockError(publicRedirect), false);
|
|
});
|
|
|
|
test("isSecurityBlockError: private-host redirect block IS a security block", () => {
|
|
const privateRedirect = new SafeOutboundFetchError("Redirect blocked", {
|
|
code: "REDIRECT_BLOCKED",
|
|
url: "https://api.example.com/probe",
|
|
method: "GET",
|
|
attempts: 1,
|
|
status: 302,
|
|
location: "http://169.254.169.254/latest/meta-data/",
|
|
isRetryable: false,
|
|
});
|
|
assert.equal(isSecurityBlockError(privateRedirect), true);
|
|
});
|
|
|
|
test("isSecurityBlockError: a URL-guard block remains a security block", () => {
|
|
const guardBlock = new SafeOutboundFetchError("Blocked private host", {
|
|
code: "URL_GUARD_BLOCKED",
|
|
url: "http://10.0.0.5/internal",
|
|
method: "GET",
|
|
attempts: 1,
|
|
isRetryable: false,
|
|
});
|
|
assert.equal(isSecurityBlockError(guardBlock), true);
|
|
});
|
|
|
|
// ─── huggingface validator (whoami-v2 auth probe) ────────────────────────────
|
|
// Fine-grained HF Inference-Provider tokens are valid even when model/task
|
|
// endpoints reject them. The validator must probe whoami-v2 as a pure auth
|
|
// check: only 401/403 is invalid; any other non-OK status is transient.
|
|
|
|
test("huggingface validator accepts a token whoami-v2 recognizes", async () => {
|
|
const calls: { url: string; headers: Record<string, string> }[] = [];
|
|
globalThis.fetch = async (url, init = {}) => {
|
|
calls.push({ url: String(url), headers: toPlainHeaders(init.headers) });
|
|
return new Response(JSON.stringify({ name: "hf-user", auth: { type: "access_token" } }), {
|
|
status: 200,
|
|
});
|
|
};
|
|
|
|
const result = await validateProviderApiKey({ provider: "huggingface", apiKey: "hf_validtoken" });
|
|
|
|
assert.equal(result.valid, true);
|
|
assert.equal(result.error, null);
|
|
assert.equal(result.method, "huggingface_whoami");
|
|
assert.equal(calls.length, 1);
|
|
assert.equal(calls[0].url, "https://huggingface.co/api/whoami-v2");
|
|
assert.equal(calls[0].headers.Authorization, "Bearer hf_validtoken");
|
|
});
|
|
|
|
test("huggingface validator treats 401/403 as an invalid token", async () => {
|
|
globalThis.fetch = async () => new Response(JSON.stringify({ error: "Unauthorized" }), { status: 401 });
|
|
const unauthorized = await validateProviderApiKey({ provider: "huggingface", apiKey: "hf_bad" });
|
|
assert.equal(unauthorized.valid, false);
|
|
assert.equal(unauthorized.error, "Invalid API key");
|
|
|
|
globalThis.fetch = async () => new Response(JSON.stringify({ error: "Forbidden" }), { status: 403 });
|
|
const forbidden = await validateProviderApiKey({ provider: "huggingface", apiKey: "hf_bad" });
|
|
assert.equal(forbidden.valid, false);
|
|
assert.equal(forbidden.error, "Invalid API key");
|
|
});
|
|
|
|
test("huggingface validator does NOT mark a fine-grained token invalid on a non-auth status", async () => {
|
|
// This is the false-negative the port fixes: a 503/404 from a model/task
|
|
// probe used to read as "invalid key". whoami-v2 returning a non-auth,
|
|
// non-OK status must surface as a transient error, never "Invalid API key".
|
|
globalThis.fetch = async () => new Response("upstream down", { status: 503 });
|
|
|
|
const result = await validateProviderApiKey({ provider: "huggingface", apiKey: "hf_finegrained" });
|
|
|
|
assert.equal(result.valid, false);
|
|
assert.notEqual(result.error, "Invalid API key");
|
|
assert.match(result.error || "", /HuggingFace token check returned 503/);
|
|
});
|