mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-14 03:02:14 +03:00
Merge branch 'pr/9006' into feat/personal-build
This commit is contained in:
@@ -0,0 +1 @@
|
||||
- **fix(executors):** Vertex AI now routes Claude models through the native Anthropic `rawPredict` endpoint instead of the generic OpenAI-compatible partner endpoint, and synthesizes a real streaming response so Claude-via-Vertex works with `stream: true` ([#8909](https://github.com/diegosouzapw/OmniRoute/pull/8909)) — thanks @wgordon17
|
||||
14
changelog.d/fixes/9006-vertex-claude-catalog-dispatch.md
Normal file
14
changelog.d/fixes/9006-vertex-claude-catalog-dispatch.md
Normal file
@@ -0,0 +1,14 @@
|
||||
- **fix(sse):** Claude reasoning-effort suffix ids (`-high`/`-low`/`-medium`/`-xhigh`) now strip
|
||||
correctly on any provider serving a real Claude model, not just the direct Anthropic provider
|
||||
([#9006](https://github.com/diegosouzapw/OmniRoute/pull/9006))
|
||||
- **fix(sse):** the no-thinking (`no-think/`) catalog variant's provider-qualification bug — which
|
||||
made it unusable outside the direct provider, both in the discovery catalog and the dashboard
|
||||
playground — is fixed ([#9006](https://github.com/diegosouzapw/OmniRoute/pull/9006))
|
||||
- **fix(sse):** a single unrecognized model id on a Vertex connection no longer cools down every
|
||||
other model on that connection for 2 minutes — Vertex 404s are now scoped to a per-model
|
||||
lockout via `passthroughModels` instead of a connection-wide cooldown
|
||||
([#9006](https://github.com/diegosouzapw/OmniRoute/pull/9006))
|
||||
- **fix(sse):** Vertex `PERMISSION_DENIED` 403s are now disambiguated using Google's own
|
||||
documented error format — a genuinely connection-wide cause (API disabled, project-level IAM
|
||||
denial) still cools the whole connection, while a model-specific denial locks out only that
|
||||
model ([#9006](https://github.com/diegosouzapw/OmniRoute/pull/9006))
|
||||
@@ -1,4 +1,5 @@
|
||||
{
|
||||
"_rebaseline_2026_07_30_9006_vertex_claude_catalog_dispatch": "PR #9006 (fix/vertex-claude-catalog-dispatch): three files, two causes. (1) src/sse/handlers/chat.ts 1845->1846 (+1): NOT this PR's own growth — this PR never touches chat.ts at all. Measured 1846 (split(\"\\n\").length) at this PR's own merge-base (before any of its 11 commits), so the drift was already inherited from already-merged PRs on release/v3.8.50 (fast-gates PR->release do not run check:file-size, same root cause as _rebaseline_2026_07_25_v3849_basered_filesize and _rebaseline_2026_07_02_5798_release_green) — no offending branch left to fix. (2) src/sse/services/auth.ts 2508->2512 (+4 net, after extraction — see below) and open-sse/handlers/chatCore.ts 5020->5023 (+3, comment-only): genuine own growth. auth.ts adds Vertex 403 PERMISSION_DENIED disambiguation (Google's google.rpc.ErrorInfo proto distinguishes a connection-wide cause — SERVICE_DISABLED, or IAM_PERMISSION_DENIED against a project-level resource — from a model-specific one scoped to a .../models/<id> resource), added mid-PR after a quality-gate reviewer flagged the plan's originally-accepted \"Vertex 403 always -> per-model lockout\" trade-off. The actual classification logic (~40 lines) was EXTRACTED into a new leaf module src/sse/services/vertexErrorClassifier.ts (mirrors the googApiKeyAuth.ts precedent, _rebaseline_2026_07_14_7034_goog_api_key), leaving only the irreducible call-site wiring in the frozen file: a 1-line import plus widening the existing #3027 per-model-403 guard condition. chatCore.ts's +3 is a pure comment expansion (no functional change) clarifying that the adjacent effort-suffix strip is no longer unconditional for every provider, requested by a separate quality-gate code-reviewer finding; not extractable (it's a comment). Auth.ts's disambiguation logic covered by 3 new test cases in tests/unit/vertex-passthrough-model-lockout.test.ts (SERVICE_DISABLED, IAM_PERMISSION_DENIED+model-resource, IAM_PERMISSION_DENIED+project-resource) plus a 4th regression test for a multi-detail-body correlation bug (reason and resource must be read from the SAME ErrorInfo detail, not independently regexed across the whole body) found by an adversarial quality-gate pass and fixed before merge.",
|
||||
"_rebaseline_2026_07_24_8470_hyperagent_sticky_thread": "PR #8470 (artickc, fix/hyperagent-tool-loop-thread-sticky) own growth: open-sse/executors/hyperagent.ts 936->1025 (wc -l; check-file-size.mjs counts via split(\"\\n\").length so the gate sees 937->1026, +89, crosses the 1000 cap). Fixes a real bug where a reverse-conversion proxy (text-Intent/JSON to Claude Code native tool_calls) rewrites assistant messages between agentic tool-loop turns, breaking HyperAgent's conversation-prefix fingerprint and cold-starting the thread mid tool-loop. Adds Anthropic tool_use/tool_result flattening to extractMessageText() plus a new rootUserFingerprint()/root-key lookup tier in resolveHyperAgentThreadBinding()/storeHyperAgentThreadAfterTurn() so the thread stays sticky across the tool loop. Cohesive additions inside the existing single-file executor; not extractable without splitting the executor mid-request-flow. Covered by tests/unit/executor-hyperagent.test.ts (19/19, +5 new cases for tool_result/tool_use flattening + root-key stickiness). Pre-merge review flagged a cross-conversation root-key collision risk (tracked in the PR's own mandatory pre-merge checklist, not yet addressed) — unrelated to this file-size ratchet, tracked separately by /fix-prs.",
|
||||
"_rebaseline_2026_07_25_8494_capability_filter_fail_closed": "PR #8494 (fix/capability-filters-fail-closed, #8488) own growth: open-sse/services/combo.ts 3640->3693 (+53) adds a fail-closed guard after filterTargetsByRequestCompatibility() — when every eligible target is excluded by request-capability filtering (vision/tools/etc) instead of quota/health, the combo now returns an explicit `capability_mismatch` 400 (describeCapabilityFilterExhaustion, imported from combo/comboStructure.ts) rather than silently falling through to a generic no-targets error, plus a `compatFilterFailOpen` escape hatch (combo config OR settings) mirrored at both the main/auto and round-robin call sites for symmetry. combo/comboStructure.ts (previously under cap, un-frozen) grows 794->918 (+124) — new home for describeCapabilityFilterExhaustion + providerSupportsEmulatedToolCalling (#5240 emulated tool-calling exemption so fail-closed does not regress prompt-emulation-only combos like all-chatgpt-web). Irreducible orchestration wiring at the existing filter chokepoint (same precedent as #7301's universal-cooldown-retry generalization). Companion test tests/unit/combo-routing-engine.test.ts 3409->3449 (+40, fail-closed/fail-open coverage across both call sites) also rebaselined. Covered by tests/unit/8488-capability-filter-fail-closed.test.ts (new) + 95/95 passing across both files. Structural shrink of combo.ts tracked in #3501.",
|
||||
"_rebaseline_2026_07_25_8499_ts7_result_union_predicates": "PR #8499 (backryun, chore/ts7-types-executor-scattered) own growth: muse-spark-web.ts 1396->1405 (+9, irreducible). Under this workspace's `strictNullChecks: false`, the boolean-literal discriminant on `GraphqlResult` (`{ ok: true } | { ok: false; error: string }`) narrows the positive `.ok===true` branch but leaves `!result.ok` at the full union under TS7, making `.error` unreachable to the checker at the two call sites (warmup, mode-switch). Fixed by adding a single `isGraphqlFailure()` type-predicate helper (doc comment + 3-line body) reused at both call sites instead of duplicating the predicate inline — not extractable to a shared module without splitting a single-file executor's local narrowing helper out of its own file. Covered by the existing muse-spark-web executor test suite (no behavior change, pure narrowing fix).",
|
||||
@@ -349,7 +350,7 @@
|
||||
"open-sse/executors/deepseek-web.ts": 1148,
|
||||
"open-sse/executors/grok-web.ts": 1044,
|
||||
"open-sse/executors/muse-spark-web.ts": 1405,
|
||||
"open-sse/handlers/chatCore.ts": 5020,
|
||||
"open-sse/handlers/chatCore.ts": 5023,
|
||||
"open-sse/handlers/imageGeneration.ts": 3101,
|
||||
"open-sse/handlers/responseSanitizer.ts": 1115,
|
||||
"open-sse/handlers/search.ts": 1536,
|
||||
@@ -400,8 +401,8 @@
|
||||
"src/shared/components/RequestLoggerV2.tsx": 1629,
|
||||
"src/shared/components/analytics/charts.tsx": 1035,
|
||||
"src/shared/services/cliRuntime.ts": 1122,
|
||||
"src/sse/handlers/chat.ts": 1845,
|
||||
"src/sse/services/auth.ts": 2508,
|
||||
"src/sse/handlers/chat.ts": 1846,
|
||||
"src/sse/services/auth.ts": 2512,
|
||||
"tests/unit/account-fallback-service.test.ts": 1572,
|
||||
"tests/unit/provider-validation-specialty.test.ts": 2980,
|
||||
"open-sse/executors/hyperagent.ts": 1026
|
||||
|
||||
@@ -30,4 +30,5 @@ export const vertexProvider: RegistryEntry = {
|
||||
{ id: "claude-opus-4-7", name: "Claude Opus 4.7 (Vertex)" },
|
||||
{ id: "claude-sonnet-4-6", name: "Claude Sonnet 4.6 (Vertex)" },
|
||||
],
|
||||
passthroughModels: true,
|
||||
};
|
||||
|
||||
@@ -138,13 +138,149 @@ function isPartnerModel(model: string) {
|
||||
return [...PARTNER_MODELS].some((prefix) => normalizedModel.startsWith(prefix));
|
||||
}
|
||||
|
||||
// Anthropic models need their own branch: they use Vertex's native Anthropic Messages API
|
||||
// (publishers/anthropic/.../rawPredict), not the generic OpenAI-compatible partner endpoint the
|
||||
// other PARTNER_MODELS entries (DeepSeek, Qwen, Llama, Mistral, GLM) go through — the OpenAI-shaped
|
||||
// endpoint 404s/"malformed argument"s for Claude models on at least some projects.
|
||||
function isClaudeModel(model: string) {
|
||||
return model.toLowerCase().startsWith("claude-");
|
||||
}
|
||||
|
||||
// Defensive normalizer: target-format resolution for manually-added custom Claude models under
|
||||
// "vertex"/"vertex-partner" was observed sending a Gemini-shaped body (contents/parts) to the
|
||||
// Anthropic rawPredict endpoint instead of the configured "claude" format, causing a hard
|
||||
// "messages: Field required" error upstream regardless of the stored per-model targetFormat. This
|
||||
// converts a Gemini-shaped body to Anthropic Messages shape so the executor works either way,
|
||||
// independent of that unresolved upstream resolution gap.
|
||||
function toAnthropicBody(body: Record<string, unknown>): Record<string, unknown> {
|
||||
const contents = body.contents as Array<{ role?: string; parts?: Array<{ text?: string }> }> | undefined;
|
||||
if (!Array.isArray(contents)) return body;
|
||||
|
||||
const messages = contents.map((c) => ({
|
||||
role: c.role === "model" ? "assistant" : "user",
|
||||
content: (c.parts || []).map((p) => p.text || "").join(""),
|
||||
}));
|
||||
const generationConfig = body.generationConfig as { maxOutputTokens?: number } | undefined;
|
||||
const systemInstruction = body.systemInstruction as { parts?: Array<{ text?: string }> } | undefined;
|
||||
|
||||
const converted: Record<string, unknown> = {
|
||||
messages,
|
||||
max_tokens: generationConfig?.maxOutputTokens || 4096,
|
||||
};
|
||||
if (systemInstruction?.parts?.length) {
|
||||
converted.system = systemInstruction.parts.map((p) => p.text || "").join("");
|
||||
}
|
||||
return converted;
|
||||
}
|
||||
|
||||
// rawPredict always returns a single complete JSON body, never real SSE framing (see buildUrl).
|
||||
// When the caller actually requested a stream, synthesize a genuine Anthropic-native event
|
||||
// sequence from that JSON so the existing claude-to-openai.ts (and sibling) response translators
|
||||
// — which already parse real message_start/content_block_*/message_delta/message_stop events —
|
||||
// can consume it correctly, instead of relying on the OpenAI-`choices`-only JSON→SSE fallback
|
||||
// (open-sse/utils/jsonToSse.ts) which cannot represent Anthropic's native response shape at all.
|
||||
function synthesizeClaudeSse(response: Record<string, unknown>): string {
|
||||
const messageId = typeof response.id === "string" ? response.id : `msg_${Date.now()}`;
|
||||
const model = typeof response.model === "string" ? response.model : "";
|
||||
const usage = (response.usage as Record<string, unknown>) || {};
|
||||
const stopReason = typeof response.stop_reason === "string" ? response.stop_reason : "end_turn";
|
||||
const stopSequence = (response.stop_sequence as string | null | undefined) ?? null;
|
||||
const content = Array.isArray(response.content) ? response.content : [];
|
||||
|
||||
const events: Array<{ event: string; data: Record<string, unknown> }> = [];
|
||||
|
||||
events.push({
|
||||
event: "message_start",
|
||||
data: {
|
||||
type: "message_start",
|
||||
message: {
|
||||
id: messageId,
|
||||
type: "message",
|
||||
role: "assistant",
|
||||
content: [],
|
||||
model,
|
||||
stop_reason: null,
|
||||
stop_sequence: null,
|
||||
usage: { input_tokens: usage.input_tokens || 0, output_tokens: 0 },
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
content.forEach((block: Record<string, unknown>, index: number) => {
|
||||
if (block.type === "text") {
|
||||
events.push({
|
||||
event: "content_block_start",
|
||||
data: { type: "content_block_start", index, content_block: { type: "text", text: "" } },
|
||||
});
|
||||
if (block.text) {
|
||||
events.push({
|
||||
event: "content_block_delta",
|
||||
data: {
|
||||
type: "content_block_delta",
|
||||
index,
|
||||
delta: { type: "text_delta", text: block.text },
|
||||
},
|
||||
});
|
||||
}
|
||||
events.push({ event: "content_block_stop", data: { type: "content_block_stop", index } });
|
||||
} else if (block.type === "tool_use") {
|
||||
events.push({
|
||||
event: "content_block_start",
|
||||
data: {
|
||||
type: "content_block_start",
|
||||
index,
|
||||
content_block: { type: "tool_use", id: block.id, name: block.name, input: {} },
|
||||
},
|
||||
});
|
||||
events.push({
|
||||
event: "content_block_delta",
|
||||
data: {
|
||||
type: "content_block_delta",
|
||||
index,
|
||||
delta: { type: "input_json_delta", partial_json: JSON.stringify(block.input ?? {}) },
|
||||
},
|
||||
});
|
||||
events.push({ event: "content_block_stop", data: { type: "content_block_stop", index } });
|
||||
} else if (block.type === "thinking") {
|
||||
events.push({
|
||||
event: "content_block_start",
|
||||
data: { type: "content_block_start", index, content_block: { type: "thinking", thinking: "" } },
|
||||
});
|
||||
if (block.thinking) {
|
||||
events.push({
|
||||
event: "content_block_delta",
|
||||
data: {
|
||||
type: "content_block_delta",
|
||||
index,
|
||||
delta: { type: "thinking_delta", thinking: block.thinking },
|
||||
},
|
||||
});
|
||||
}
|
||||
events.push({ event: "content_block_stop", data: { type: "content_block_stop", index } });
|
||||
}
|
||||
});
|
||||
|
||||
events.push({
|
||||
event: "message_delta",
|
||||
data: {
|
||||
type: "message_delta",
|
||||
delta: { stop_reason: stopReason, stop_sequence: stopSequence },
|
||||
usage: { output_tokens: usage.output_tokens || 0 },
|
||||
},
|
||||
});
|
||||
|
||||
events.push({ event: "message_stop", data: { type: "message_stop" } });
|
||||
|
||||
return events.map((e) => `event: ${e.event}\ndata: ${JSON.stringify(e.data)}\n\n`).join("");
|
||||
}
|
||||
|
||||
export class VertexExecutor extends BaseExecutor {
|
||||
constructor() {
|
||||
super("vertex", PROVIDERS.vertex);
|
||||
}
|
||||
|
||||
async execute(input: ExecuteInput) {
|
||||
const { credentials, log } = input;
|
||||
const { credentials, log, model, stream } = input;
|
||||
// Defensive: trim stray surrounding whitespace from a pasted credential.
|
||||
if (typeof credentials.apiKey === "string") {
|
||||
credentials.apiKey = credentials.apiKey.trim();
|
||||
@@ -160,7 +296,53 @@ export class VertexExecutor extends BaseExecutor {
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
return super.execute(input);
|
||||
if (isClaudeModel(model) && input.body && typeof input.body === "object") {
|
||||
let body = input.body as Record<string, unknown>;
|
||||
if (!Array.isArray(body.messages)) {
|
||||
body = toAnthropicBody(body);
|
||||
input.body = body;
|
||||
}
|
||||
// The rawPredict endpoint requires "anthropic_version" in the body (Vertex's substitute
|
||||
// for the "anthropic-version" header used by Anthropic's direct API).
|
||||
body.anthropic_version ??= "vertex-2023-10-16";
|
||||
// Unlike Anthropic's direct API (which reads the model from the body), Vertex's
|
||||
// rawPredict endpoint already encodes project/region/model in the URL and 400s with
|
||||
// "model: Extra inputs are not permitted" if the translated request body still carries
|
||||
// one (the openai→claude request translator copies the client's model field over).
|
||||
delete body.model;
|
||||
}
|
||||
|
||||
const result = await super.execute(input);
|
||||
|
||||
if (isClaudeModel(model) && stream) {
|
||||
const response = result instanceof Response ? result : result?.response;
|
||||
if (response?.ok) {
|
||||
const contentType = response.headers.get("content-type") || "";
|
||||
if (contentType.includes("application/json") && !contentType.includes("text/event-stream")) {
|
||||
const jsonText = await response.text();
|
||||
let newBody = jsonText;
|
||||
let newContentType = contentType;
|
||||
try {
|
||||
newBody = synthesizeClaudeSse(JSON.parse(jsonText));
|
||||
newContentType = "text/event-stream";
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err);
|
||||
log?.warn?.("VERTEX", `Failed to synthesize Claude SSE stream: ${message}`);
|
||||
}
|
||||
const newHeaders = new Headers(response.headers);
|
||||
newHeaders.set("content-type", newContentType);
|
||||
newHeaders.delete("content-length");
|
||||
const newResponse = new Response(newBody, {
|
||||
status: response.status,
|
||||
statusText: response.statusText,
|
||||
headers: newHeaders,
|
||||
});
|
||||
return result instanceof Response ? newResponse : { ...result, response: newResponse };
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
buildUrl(model: string, stream: boolean, urlIndex = 0, credentials: any = null) {
|
||||
@@ -189,6 +371,13 @@ export class VertexExecutor extends BaseExecutor {
|
||||
}
|
||||
}
|
||||
|
||||
if (isClaudeModel(model)) {
|
||||
// streamRawPredict?alt=sse was verified to return a single plain JSON body (not real SSE
|
||||
// framing) rather than actual chunked events, which breaks the SSE parser upstream
|
||||
// ("stream ended before producing a non-ping SSE event"). rawPredict is confirmed reliable
|
||||
// for both streaming and non-streaming requests; always use it here.
|
||||
return `https://aiplatform.googleapis.com/v1/projects/${project}/locations/${region}/publishers/anthropic/models/${model}:rawPredict`;
|
||||
}
|
||||
if (isPartnerModel(model)) {
|
||||
return `https://aiplatform.googleapis.com/v1/projects/${project}/locations/global/endpoints/openapi/chat/completions`;
|
||||
}
|
||||
|
||||
@@ -731,7 +731,10 @@ export async function handleChatCore({
|
||||
// wins; native Claude passthrough is left untouched (it carries its own `thinking`),
|
||||
// and non-thinking base models are cleaned up later by normalizeThinkingForModel().
|
||||
// Extracted to chatCore/claudeEffortVariant.ts (#3501); mutates body in place and returns the
|
||||
// stripped model + an optional log line, keeping behaviour byte-identical.
|
||||
// stripped model + an optional log line. The strip is unconditional (byte-identical to the
|
||||
// original behavior) for the claude/Claude-Code-compatible lane; for any other provider it
|
||||
// additionally requires isKnownClaudeEffortBaseModel(baseModel) to verify the base id is a
|
||||
// real, effort-capable Claude model before stripping (vertex-claude-catalog-dispatch fix).
|
||||
{
|
||||
const effortVariant = applyClaudeEffortVariant({
|
||||
provider,
|
||||
|
||||
@@ -15,6 +15,7 @@
|
||||
import { splitClaudeEffortSuffix } from "../../config/providerModels.ts";
|
||||
import { isClaudeCodeCompatibleProvider } from "../../services/claudeCodeCompatible.ts";
|
||||
import { FORMATS } from "../../translator/formats.ts";
|
||||
import { isKnownClaudeEffortBaseModel } from "../../utils/claudeEffortVariants.ts";
|
||||
|
||||
/**
|
||||
* True when the client already supplied an explicit reasoning effort (top-level reasoning_effort,
|
||||
@@ -40,12 +41,10 @@ export function applyClaudeEffortVariant(opts: {
|
||||
let effectiveModel = opts.effectiveModel;
|
||||
let log: string | null = null;
|
||||
|
||||
if (
|
||||
(provider === "claude" || isClaudeCodeCompatibleProvider(provider)) &&
|
||||
typeof effectiveModel === "string"
|
||||
) {
|
||||
if (typeof effectiveModel === "string") {
|
||||
const { baseModel, effort } = splitClaudeEffortSuffix(effectiveModel);
|
||||
if (effort) {
|
||||
const isDirectClaudeLane = provider === "claude" || isClaudeCodeCompatibleProvider(provider);
|
||||
if (effort && (isDirectClaudeLane || isKnownClaudeEffortBaseModel(baseModel))) {
|
||||
effectiveModel = baseModel;
|
||||
if (body && typeof body === "object" && !Array.isArray(body)) {
|
||||
const claudeBody = body as Record<string, unknown>;
|
||||
|
||||
@@ -33,7 +33,7 @@ export const CC_DISCOVERY_COMBO_PREFIX = "claude/combo/";
|
||||
// Ids that already live under the claude/anthropic namespace — never re-mirror them.
|
||||
const ALREADY_CLAUDE_RE = /^(?:claude|anthropic)(?:\/|$)/i;
|
||||
// Ids that already carry a reasoning-effort suffix — v1 only mirrors base ids.
|
||||
const EFFORT_SUFFIX_RE = /-(?:xhigh|high|medium|low)$/i;
|
||||
const CLAUDE_EFFORT_SUFFIX_RE = /-(?:xhigh|high|medium|low)$/i;
|
||||
const NO_THINKING_PREFIX = "no-think/";
|
||||
// Built-in `auto`/`auto/*` combos are synthesized by createBuiltinAutoCombo, NOT
|
||||
// stored in the DB combos table — the request-path resolver (getComboByName) can't
|
||||
@@ -66,7 +66,14 @@ function isMirrorableId(id: string): boolean {
|
||||
if (id.length === 0) return false;
|
||||
if (ALREADY_CLAUDE_RE.test(id)) return false;
|
||||
if (id.startsWith(NO_THINKING_PREFIX)) return false;
|
||||
return !EFFORT_SUFFIX_RE.test(id);
|
||||
return !CLAUDE_EFFORT_SUFFIX_RE.test(id);
|
||||
}
|
||||
|
||||
/** Strip a `<provider>/` prefix to get the bare model name, matching the convention in
|
||||
* claudeEffortVariants.ts / noThinkingAlias.ts. */
|
||||
function bareModelName(id: string): string {
|
||||
const slash = id.lastIndexOf("/");
|
||||
return slash >= 0 ? id.slice(slash + 1) : id;
|
||||
}
|
||||
|
||||
export function appendCcDiscoveryAliases<T extends CcDiscoveryCatalogEntry>(
|
||||
@@ -90,7 +97,10 @@ export function appendCcDiscoveryAliases<T extends CcDiscoveryCatalogEntry>(
|
||||
aliases.push({
|
||||
...model,
|
||||
id: aliasId,
|
||||
root: id,
|
||||
// Combo names may legally contain "/" (comboNameSchema allows it), so a combo's
|
||||
// root must stay the full name verbatim — only real provider-qualified ids get
|
||||
// the "/" stripped down to the bare model name.
|
||||
root: isCombo ? id : bareModelName(id),
|
||||
display_name: `${label} (OmniRoute)`,
|
||||
} as T);
|
||||
}
|
||||
|
||||
@@ -64,6 +64,17 @@ export function formatClaudeEffortLabel(level: string): string {
|
||||
return level.charAt(0).toUpperCase() + level.slice(1);
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether `bareModelId` (no provider prefix, no effort suffix) is a real,
|
||||
* effort-capable Claude-family model — the single source of truth used both to
|
||||
* decide whether the catalog should advertise an effort variant AND whether
|
||||
* dispatch-time stripping should unwind one back to this model.
|
||||
*/
|
||||
export function isKnownClaudeEffortBaseModel(bareModelId: string): boolean {
|
||||
const spec = getModelSpec(bareModelId);
|
||||
return spec?.supportsThinking === true && CLAUDE_NAME_RE.test(bareModelId);
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the catalog should advertise reasoning-effort variants for this entry.
|
||||
*
|
||||
@@ -84,10 +95,7 @@ export function shouldExposeClaudeEffortVariants(
|
||||
if (CLAUDE_EFFORT_SUFFIX_RE.test(id)) return false;
|
||||
|
||||
const name = bareModelName(id);
|
||||
const spec = getModelSpec(name);
|
||||
if (!spec) return false;
|
||||
|
||||
return spec.supportsThinking === true && CLAUDE_NAME_RE.test(name);
|
||||
return isKnownClaudeEffortBaseModel(name);
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -31,6 +31,15 @@ import { getModelSpec } from "@/shared/constants/modelSpecs";
|
||||
|
||||
export const NO_THINKING_PREFIX = "no-think/";
|
||||
|
||||
// Ids that already carry a Claude reasoning-effort suffix (see
|
||||
// claudeEffortVariants.ts's identical constant) — a no-think variant of an effort
|
||||
// variant would combine two independent OmniRoute catalog conventions on the same
|
||||
// id. Dispatch-time, applyNoThinkingAlias pre-sets reasoning_effort:"none" before
|
||||
// applyClaudeEffortVariant's hasExplicitClaudeEffort() check runs, so the pre-set
|
||||
// "none" is treated as explicit and the suffix's implied effort is silently
|
||||
// discarded — semantically incoherent, so never advertise the combination.
|
||||
const CLAUDE_EFFORT_SUFFIX_RE = /-(?:xhigh|high|medium|low)$/i;
|
||||
|
||||
/** True when `modelId` carries the no-thinking gateway prefix. */
|
||||
export function isNoThinkingAlias(modelId: unknown): modelId is string {
|
||||
return typeof modelId === "string" && modelId.startsWith(NO_THINKING_PREFIX);
|
||||
@@ -108,6 +117,7 @@ export function shouldExposeNoThinkingAlias(model: CatalogModelEntry): boolean {
|
||||
if (typeof id !== "string" || id.length === 0) return false;
|
||||
if (model.owned_by === "combo") return false; // combos are virtual
|
||||
if (isNoThinkingAlias(id)) return false; // never double-alias
|
||||
if (CLAUDE_EFFORT_SUFFIX_RE.test(id)) return false; // never combine with an effort-suffix id
|
||||
|
||||
const name = bareModelName(id);
|
||||
const spec = getModelSpec(name);
|
||||
@@ -158,7 +168,8 @@ export function appendNoThinkingVariants<T extends CatalogModelEntry>(
|
||||
const rawId = model.id as string;
|
||||
const qualifiedId = aliasToCanonical ? normalizeProviderPrefix(rawId, aliasToCanonical) : rawId;
|
||||
const aliasId = toNoThinkingAlias(qualifiedId);
|
||||
const variant: T = { ...model, id: aliasId, root: aliasId };
|
||||
const bareRoot = toNoThinkingAlias(bareModelName(qualifiedId));
|
||||
const variant: T = { ...model, id: aliasId, root: bareRoot };
|
||||
if (typeof model.name === "string" && model.name) {
|
||||
variant.name = `${model.name} (no thinking)`;
|
||||
}
|
||||
|
||||
@@ -35,6 +35,10 @@ function resolvePlaygroundKeyId(
|
||||
return keys.find((k) => k.key === selectedMaskedKey)?.id ?? null;
|
||||
}
|
||||
|
||||
// Mirrors NO_THINKING_PREFIX in open-sse/utils/noThinkingAlias.ts — kept as a local literal
|
||||
// (not imported) to avoid pulling server-side catalog modules into the client bundle.
|
||||
const NO_THINKING_PREFIX = "no-think/";
|
||||
|
||||
/**
|
||||
* Qualify a provider-scoped playground model with its routing prefix so
|
||||
* OmniRoute can resolve it unambiguously. The previous heuristic only prefixed
|
||||
@@ -52,6 +56,14 @@ export function qualifyPlaygroundModel(
|
||||
): string {
|
||||
const m = (model ?? "").trim();
|
||||
if (!m || !routingPrefix) return m;
|
||||
// A no-think id's real wire form is `no-think/<provider>/<model>` — the provider segment
|
||||
// sits AFTER the prefix, not at the front, so it needs its own qualification branch instead
|
||||
// of the generic leading-prefix check below.
|
||||
if (m.startsWith(NO_THINKING_PREFIX)) {
|
||||
const inner = m.slice(NO_THINKING_PREFIX.length);
|
||||
const alreadyQualified = inner === routingPrefix || inner.startsWith(`${routingPrefix}/`);
|
||||
return alreadyQualified ? m : `${NO_THINKING_PREFIX}${routingPrefix}/${inner}`;
|
||||
}
|
||||
return m === routingPrefix || m.startsWith(`${routingPrefix}/`) ? m : `${routingPrefix}/${m}`;
|
||||
}
|
||||
|
||||
|
||||
@@ -1445,6 +1445,7 @@ async function handleSingleModelChat(
|
||||
comboExecutionKey: runtimeOptions.comboExecutionKey ?? runtimeOptions.comboStepId ?? null,
|
||||
extendedContext,
|
||||
modelApiFormat: apiFormat,
|
||||
modelTargetFormat: targetFormat,
|
||||
providerProfile,
|
||||
cachedSettings: runtimeOptions.cachedSettings,
|
||||
skipUpstreamRetry: runtimeOptions.skipUpstreamRetry ?? false,
|
||||
|
||||
@@ -4,14 +4,8 @@ import { connectionHasExtraKeys } from "@omniroute/open-sse/services/apiKeyRotat
|
||||
import { createBuiltinAutoCombo } from "@omniroute/open-sse/services/autoCombo/builtinCatalog.ts";
|
||||
import * as log from "../utils/logger";
|
||||
import { updateProviderCredentials } from "../services/tokenRefresh";
|
||||
import {
|
||||
detectFormatFromEndpoint,
|
||||
getTargetFormat,
|
||||
} from "@omniroute/open-sse/services/provider.ts";
|
||||
import {
|
||||
getModelTargetFormat,
|
||||
PROVIDER_ID_TO_ALIAS,
|
||||
} from "@omniroute/open-sse/config/providerModels.ts";
|
||||
import { detectFormatFromEndpoint } from "@omniroute/open-sse/services/provider.ts";
|
||||
import { resolveChatCoreTargetFormat } from "@omniroute/open-sse/handlers/chatCore/targetFormat.ts";
|
||||
import { handleChatCore } from "@omniroute/open-sse/handlers/chatCore.ts";
|
||||
import {
|
||||
errorResponse,
|
||||
@@ -297,12 +291,25 @@ export async function resolveModelOrError(
|
||||
? ((modelInfo as { apiFormat?: string }).apiFormat as string)
|
||||
: undefined
|
||||
: undefined;
|
||||
const providerAlias = PROVIDER_ID_TO_ALIAS[provider] || provider;
|
||||
let targetFormat = getModelTargetFormat(providerAlias, model) || getTargetFormat(provider);
|
||||
if (apiFormat === "responses") {
|
||||
targetFormat = "openai-responses";
|
||||
log.info("ROUTING", `Custom model apiFormat=responses → targetFormat=openai-responses`);
|
||||
}
|
||||
// customModelTargetFormat: #2905 per-model wire-format override for custom models,
|
||||
// injected by getModelInfo. Must be threaded into the same resolution formula
|
||||
// chatCore.ts uses (static registry > custom-model DB override > provider default) —
|
||||
// a model that's ALSO a static registry entry (e.g. a Vertex Claude model with no
|
||||
// per-model registry targetFormat) otherwise silently drops the DB override and
|
||||
// falls through to the provider default, breaking response translation.
|
||||
const customModelTargetFormat: string | undefined =
|
||||
modelInfo && typeof modelInfo === "object" && "targetFormat" in modelInfo
|
||||
? typeof (modelInfo as { targetFormat?: unknown }).targetFormat === "string"
|
||||
? ((modelInfo as { targetFormat?: string }).targetFormat as string)
|
||||
: undefined
|
||||
: undefined;
|
||||
const { alias: providerAlias, targetFormat } = resolveChatCoreTargetFormat({
|
||||
provider,
|
||||
resolvedModel: model,
|
||||
apiFormat,
|
||||
customModelTargetFormat,
|
||||
providerSpecificData: undefined,
|
||||
});
|
||||
|
||||
const ctxTag = extendedContext && providerAlias === "claude" ? " [1m]" : "";
|
||||
if (modelStr !== `${provider}/${model}`) {
|
||||
@@ -389,6 +396,7 @@ export async function executeChatWithBreaker({
|
||||
comboExecutionKey,
|
||||
extendedContext,
|
||||
modelApiFormat,
|
||||
modelTargetFormat,
|
||||
providerProfile,
|
||||
cachedSettings,
|
||||
skipUpstreamRetry = false,
|
||||
@@ -416,7 +424,19 @@ export async function executeChatWithBreaker({
|
||||
runWithProxyContext(proxyInfo?.proxy || null, () =>
|
||||
(handleChatCore as any)({
|
||||
body: { ...body, model: `${provider}/${model}` },
|
||||
modelInfo: { provider, model, extendedContext, apiFormat: modelApiFormat },
|
||||
// #2905-followup: forward the already-resolved custom-model targetFormat
|
||||
// override through as modelInfo.targetFormat. Without this, chatCore.ts's
|
||||
// own resolveChatCoreRequestSetup() reads customModelTargetFormat off THIS
|
||||
// modelInfo object (not the one resolveModelOrError computed it from) and
|
||||
// finds nothing, silently re-deriving targetFormat from the static registry
|
||||
// / provider default and discarding the DB override a second time.
|
||||
modelInfo: {
|
||||
provider,
|
||||
model,
|
||||
extendedContext,
|
||||
apiFormat: modelApiFormat,
|
||||
targetFormat: modelTargetFormat,
|
||||
},
|
||||
credentials: refreshedCredentials,
|
||||
log: handlerLog,
|
||||
clientRawRequest,
|
||||
|
||||
@@ -77,6 +77,7 @@ import { isNoAuthProviderBlockedBySettings } from "./noAuthProviderSettings";
|
||||
import { resolveAccountProxiesFromRegistry } from "./noAuthProxyResolution";
|
||||
import { getNoAuthHydrationProviderIds } from "./noAuthProviderSiblings";
|
||||
import { getResource404Bypass } from "./requestResourceHealth";
|
||||
import { isVertexConnectionWidePermissionDenied } from "./vertexErrorClassifier";
|
||||
import * as log from "../utils/logger";
|
||||
import { fisherYatesShuffle, getNextFromDeckSync } from "@/shared/utils/shuffleDeck";
|
||||
|
||||
@@ -2146,7 +2147,14 @@ export async function markAccountUnavailable(
|
||||
: rawCooldownMs;
|
||||
|
||||
// ── #3027: per-model subscription/permission 403 → model-only lockout ──
|
||||
if (isPerModelQuotaProvider && status === 403 && provider && model && !terminalStatus) {
|
||||
if (
|
||||
isPerModelQuotaProvider &&
|
||||
status === 403 &&
|
||||
provider &&
|
||||
model &&
|
||||
!terminalStatus &&
|
||||
!(provider === "vertex" && isVertexConnectionWidePermissionDenied(errorText))
|
||||
) {
|
||||
const lockout = recordModelLockoutFailure(
|
||||
provider,
|
||||
connectionId,
|
||||
|
||||
66
src/sse/services/vertexErrorClassifier.ts
Normal file
66
src/sse/services/vertexErrorClassifier.ts
Normal file
@@ -0,0 +1,66 @@
|
||||
/**
|
||||
* Google's google.rpc.ErrorInfo proto reliably distinguishes a connection-wide
|
||||
* PERMISSION_DENIED (API not enabled, or a project-level IAM denial) from a
|
||||
* model-specific one (IAM denial scoped to a .../models/<id> resource) — see
|
||||
* https://cloud.google.com/apis/design/errors#error_info. Only returns true on
|
||||
* POSITIVE evidence of a connection-wide cause; any other shape (including a
|
||||
* missing/malformed resource field) falls through to the existing per-model
|
||||
* lockout behavior, since that's the safer default and the actual bug this
|
||||
* plan fixes (avoid defaulting BACK toward the connection-wide cooldown this
|
||||
* plan exists to avoid).
|
||||
*
|
||||
* Parses the body as JSON and inspects each ErrorInfo-shaped detail object so
|
||||
* `reason` and `resource` are correlated within the SAME detail entry — a
|
||||
* multi-detail error body (unusual but possible) must not let one detail's
|
||||
* resource leak into another detail's reason check. Falls back to a permissive
|
||||
* regex scan (pre-JSON-parsing behavior) only when the body isn't parseable
|
||||
* JSON or doesn't contain a `details` array, since Vertex error bodies aren't
|
||||
* guaranteed to always be well-formed JSON.
|
||||
*
|
||||
* Extracted to its own module so the single call site in
|
||||
* `./auth.ts::markAccountUnavailable()` stays thin wiring, without growing the
|
||||
* frozen `auth.ts` file (`config/quality/file-size-baseline.json`).
|
||||
*/
|
||||
export function isVertexConnectionWidePermissionDenied(
|
||||
errorText: string | null | undefined
|
||||
): boolean {
|
||||
if (!errorText) return false;
|
||||
|
||||
try {
|
||||
const parsed = JSON.parse(errorText);
|
||||
const details: unknown[] =
|
||||
parsed?.error?.details ?? parsed?.details ?? (Array.isArray(parsed) ? parsed : []);
|
||||
if (Array.isArray(details) && details.length > 0) {
|
||||
for (const detail of details) {
|
||||
if (!detail || typeof detail !== "object") continue;
|
||||
const reason = (detail as Record<string, unknown>).reason;
|
||||
if (reason === "SERVICE_DISABLED") return true;
|
||||
if (reason === "IAM_PERMISSION_DENIED") {
|
||||
const metadata = (detail as Record<string, unknown>).metadata;
|
||||
const resource =
|
||||
metadata && typeof metadata === "object"
|
||||
? (metadata as Record<string, unknown>).resource
|
||||
: undefined;
|
||||
if (typeof resource === "string" && !resource.includes("/models/")) return true;
|
||||
}
|
||||
}
|
||||
// Well-formed details array present but no detail matched a connection-wide
|
||||
// pattern (e.g. IAM_PERMISSION_DENIED with a /models/ resource, or no
|
||||
// recognized reason at all) — per-model lockout is correct, don't fall
|
||||
// through to the regex heuristic (it would just re-derive the same answer
|
||||
// less precisely, or worse, could false-positive on stray substrings).
|
||||
return false;
|
||||
}
|
||||
} catch {
|
||||
// Not parseable JSON — fall through to the regex heuristic below.
|
||||
}
|
||||
|
||||
// Fallback for non-JSON or unexpected-shape error bodies (regex-based,
|
||||
// pre-JSON-parsing heuristic — kept for robustness against malformed bodies).
|
||||
if (/"reason"\s*:\s*"SERVICE_DISABLED"/.test(errorText)) return true;
|
||||
if (/"reason"\s*:\s*"IAM_PERMISSION_DENIED"/.test(errorText)) {
|
||||
const resourceMatch = errorText.match(/"resource"\s*:\s*"([^"]*)"/);
|
||||
if (resourceMatch && !resourceMatch[1].includes("/models/")) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
@@ -39,9 +39,7 @@
|
||||
"incremental": true,
|
||||
"incrementalFile": "reports/mutation/stryker-incremental.json",
|
||||
"testRunner": "tap",
|
||||
"plugins": [
|
||||
"@stryker-mutator/tap-runner"
|
||||
],
|
||||
"plugins": ["@stryker-mutator/tap-runner"],
|
||||
"tap": {
|
||||
"testFiles": [
|
||||
"tests/unit/7993-noauth-proxy-routing.test.ts",
|
||||
@@ -307,7 +305,8 @@
|
||||
"tests/unit/upstream-retry-hints-toggle.test.ts",
|
||||
"tests/unit/upstream-timeout-model-override.test.ts",
|
||||
"tests/unit/usage-service-hardening.test.ts",
|
||||
"tests/unit/validate-response-quality.test.ts"
|
||||
"tests/unit/validate-response-quality.test.ts",
|
||||
"tests/unit/vertex-passthrough-model-lockout.test.ts"
|
||||
],
|
||||
"nodeArgs": [
|
||||
"--import",
|
||||
@@ -416,11 +415,7 @@
|
||||
".worktrees",
|
||||
".stryker-tmp"
|
||||
],
|
||||
"reporters": [
|
||||
"progress",
|
||||
"html",
|
||||
"json"
|
||||
],
|
||||
"reporters": ["progress", "html", "json"],
|
||||
"htmlReporter": {
|
||||
"fileName": "reports/mutation/mutation.html"
|
||||
},
|
||||
|
||||
@@ -30,11 +30,25 @@ test("adds a claude/ mirror with display_name and root for an eligible model", (
|
||||
assert.deepEqual(out[0], models[0]);
|
||||
const alias = out[1];
|
||||
assert.equal(alias.id, "claude/kimi/kimi-k2.6");
|
||||
assert.equal(alias.root, "kimi/kimi-k2.6");
|
||||
assert.equal(alias.root, "kimi-k2.6");
|
||||
assert.equal(alias.display_name, "Kimi K2.6 (OmniRoute)");
|
||||
assert.equal(alias.owned_by, "kimi");
|
||||
});
|
||||
|
||||
test("keeps root bare even when the original id carries a provider prefix", () => {
|
||||
const models: CatalogEntry[] = [
|
||||
{ id: "vertex/claude-sonnet-5", owned_by: "vertex", name: "Claude Sonnet 5 (Vertex)" },
|
||||
];
|
||||
const out = appendCcDiscoveryAliases(models, alwaysEnabled);
|
||||
const alias = out.find((m) => m.id === "claude/vertex/claude-sonnet-5");
|
||||
assert.ok(alias, "mirror entry with the fully-qualified id must exist");
|
||||
assert.equal(
|
||||
alias!.root,
|
||||
"claude-sonnet-5",
|
||||
"root must be bare, matching the no-think/effort-variant convention"
|
||||
);
|
||||
});
|
||||
|
||||
test("falls back to the id for display_name when name is missing", () => {
|
||||
const models: CatalogEntry[] = [{ id: "kimi/kimi-k2.6" }];
|
||||
const out = appendCcDiscoveryAliases(models, alwaysEnabled);
|
||||
@@ -86,6 +100,17 @@ test("mirrors combo names containing spaces (comboNameSchema allows them)", () =
|
||||
assert.equal(out[1].root, "Custo Otimizado BR");
|
||||
});
|
||||
|
||||
test("keeps a combo's root the full name verbatim when the combo name contains a slash", () => {
|
||||
// comboNameSchema (src/shared/validation/schemas/combo.ts) explicitly allows "/" in
|
||||
// combo names, so bareModelName must NOT be applied to combo entries — only to real
|
||||
// provider-qualified model ids.
|
||||
const models: CatalogEntry[] = [{ id: "Team/Alpha", owned_by: "combo", name: "Team/Alpha" }];
|
||||
const out = appendCcDiscoveryAliases(models, alwaysEnabled);
|
||||
assert.equal(out.length, 2);
|
||||
assert.equal(out[1].id, "claude/combo/Team/Alpha");
|
||||
assert.equal(out[1].root, "Team/Alpha", "root must be the full combo name, not truncated");
|
||||
});
|
||||
|
||||
test("skips disabled entries and returns the same array reference when nothing is eligible", () => {
|
||||
const models: CatalogEntry[] = [{ id: "kimi/kimi-k2.6", owned_by: "kimi" }];
|
||||
const out = appendCcDiscoveryAliases(models, () => false);
|
||||
|
||||
@@ -201,6 +201,37 @@ test("resolveModelOrError keeps bare gpt-5.5 on OpenAI when OpenAI is the only a
|
||||
assert.equal(result.model, "gpt-5.5");
|
||||
});
|
||||
|
||||
test("resolveModelOrError honors a custom-model targetFormat override even when the model id also exists in the static provider registry", async () => {
|
||||
// #8852-followup: "claude-sonnet-4-6" is a real static registry entry under
|
||||
// "vertex" (see open-sse/config/providers/registry/vertex/index.ts) with no
|
||||
// per-model targetFormat, so the provider default ("gemini") normally applies.
|
||||
// A user who manually added the same id as a custom model with an explicit
|
||||
// "claude" targetFormat override must have that override win — otherwise
|
||||
// Vertex's native Anthropic response shape gets mistranslated as Gemini's,
|
||||
// silently dropping all response content.
|
||||
await seedConnection("vertex");
|
||||
const modelsDb = await import("../../src/lib/db/models.ts");
|
||||
await modelsDb.addCustomModel(
|
||||
"vertex",
|
||||
"claude-sonnet-4-6",
|
||||
"Claude Sonnet 4.6 (Vertex)",
|
||||
"manual",
|
||||
"chat-completions",
|
||||
["chat"],
|
||||
"claude"
|
||||
);
|
||||
|
||||
const result = await resolveModelOrError(
|
||||
"vertex/claude-sonnet-4-6",
|
||||
{ model: "vertex/claude-sonnet-4-6", messages: [{ role: "user", content: "hi" }] },
|
||||
"/v1/chat/completions"
|
||||
);
|
||||
|
||||
assert.equal(result.provider, "vertex");
|
||||
assert.equal(result.model, "claude-sonnet-4-6");
|
||||
assert.equal(result.targetFormat, "claude");
|
||||
});
|
||||
|
||||
test("checkPipelineGates blocks providers with an open circuit breaker", async () => {
|
||||
const breaker = getCircuitBreaker("openai");
|
||||
breaker.state = STATE.OPEN;
|
||||
|
||||
@@ -2,8 +2,9 @@
|
||||
// Characterization of applyClaudeEffortVariant — the Claude effort-suffix normalization extracted
|
||||
// from handleChatCore (chatCore god-file decomposition, #3501). The VS Code "Effort" slider
|
||||
// advertises claude-...-{low,medium,high,xhigh,max}; Anthropic has no such model, so the suffix is
|
||||
// stripped to the base id and surfaced as reasoning_effort. Locks: the provider gate (claude /
|
||||
// claude-code-compatible only), the in-place body mutation (model + reasoning_effort), the
|
||||
// stripped to the base id and surfaced as reasoning_effort. Locks: the direct-Claude-lane
|
||||
// unconditional strip (claude / claude-code-compatible), the predicate-gated strip for any other
|
||||
// provider serving a real Claude model, the in-place body mutation (model + reasoning_effort), the
|
||||
// sourceFormat==="claude" skip, the explicit-effort-wins rule, and the returned effectiveModel/log.
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
@@ -51,7 +52,11 @@ test("sourceFormat 'claude' strips the model but does NOT inject reasoning_effor
|
||||
});
|
||||
|
||||
test("an explicit client reasoning_effort wins (not overwritten)", () => {
|
||||
const body: Record<string, unknown> = { model: "claude-sonnet-4-low", reasoning_effort: "high", messages: [] };
|
||||
const body: Record<string, unknown> = {
|
||||
model: "claude-sonnet-4-low",
|
||||
reasoning_effort: "high",
|
||||
messages: [],
|
||||
};
|
||||
const r = applyClaudeEffortVariant({
|
||||
provider: "claude",
|
||||
effectiveModel: "claude-sonnet-4-low",
|
||||
@@ -105,3 +110,66 @@ test("non-claude provider is a no-op even with an effort suffix", () => {
|
||||
assert.equal(body.reasoning_effort, undefined);
|
||||
assert.equal(r.log, null);
|
||||
});
|
||||
|
||||
test("non-claude provider serving a real Claude model strips the effort suffix", () => {
|
||||
const body: Record<string, unknown> = { model: "claude-sonnet-5-high", messages: [] };
|
||||
const r = applyClaudeEffortVariant({
|
||||
provider: "vertex",
|
||||
effectiveModel: "claude-sonnet-5-high",
|
||||
body,
|
||||
sourceFormat: FORMATS.OPENAI,
|
||||
});
|
||||
assert.equal(r.effectiveModel, "claude-sonnet-5");
|
||||
assert.equal(body.model, "claude-sonnet-5");
|
||||
assert.equal(body.reasoning_effort, "high");
|
||||
});
|
||||
|
||||
test("safety guard: non-claude provider with a non-Claude model ending in a suffix word is left unchanged", () => {
|
||||
const body: Record<string, unknown> = { model: "custom-model-high", messages: [] };
|
||||
const r = applyClaudeEffortVariant({
|
||||
provider: "some-other-provider",
|
||||
effectiveModel: "custom-model-high",
|
||||
body,
|
||||
sourceFormat: FORMATS.OPENAI,
|
||||
});
|
||||
assert.equal(r.effectiveModel, "custom-model-high");
|
||||
assert.equal(body.model, "custom-model-high");
|
||||
assert.equal(body.reasoning_effort, undefined);
|
||||
assert.equal(r.log, null);
|
||||
});
|
||||
|
||||
test("claude-code-compatible provider strips even an unregistered model id (direct lane short-circuits the predicate)", () => {
|
||||
// Proves the "unconditional strip, zero regression" claim: isDirectClaudeLane short-circuits
|
||||
// the `||`, so isKnownClaudeEffortBaseModel() is never consulted for claude/CC-compatible
|
||||
// providers — unlike the safety-guard case above, which requires the predicate to pass.
|
||||
const body: Record<string, unknown> = {
|
||||
model: "totally-unregistered-model-xyz-high",
|
||||
messages: [],
|
||||
};
|
||||
const r = applyClaudeEffortVariant({
|
||||
provider: "anthropic-compatible-cc-default",
|
||||
effectiveModel: "totally-unregistered-model-xyz-high",
|
||||
body,
|
||||
sourceFormat: FORMATS.OPENAI,
|
||||
});
|
||||
assert.equal(r.effectiveModel, "totally-unregistered-model-xyz");
|
||||
assert.equal(body.model, "totally-unregistered-model-xyz");
|
||||
assert.equal(body.reasoning_effort, "high");
|
||||
});
|
||||
|
||||
test("no-think alias's explicit reasoning_effort:none is not overwritten by a stripped effort suffix", () => {
|
||||
const body: Record<string, unknown> = {
|
||||
model: "claude-sonnet-5-high",
|
||||
reasoning_effort: "none",
|
||||
messages: [],
|
||||
};
|
||||
const r = applyClaudeEffortVariant({
|
||||
provider: "vertex",
|
||||
effectiveModel: "claude-sonnet-5-high",
|
||||
body,
|
||||
sourceFormat: FORMATS.OPENAI,
|
||||
});
|
||||
assert.equal(r.effectiveModel, "claude-sonnet-5");
|
||||
assert.equal(body.model, "claude-sonnet-5");
|
||||
assert.equal(body.reasoning_effort, "none");
|
||||
});
|
||||
|
||||
@@ -6,9 +6,12 @@ import {
|
||||
CLAUDE_XHIGH_EFFORT_LEVEL,
|
||||
formatClaudeEffortLabel,
|
||||
shouldExposeClaudeEffortVariants,
|
||||
isKnownClaudeEffortBaseModel,
|
||||
claudeEffortLevelsFor,
|
||||
appendClaudeEffortVariants,
|
||||
} from "../../open-sse/utils/claudeEffortVariants.ts";
|
||||
import { shouldExposeNoThinkingAlias } from "../../open-sse/utils/noThinkingAlias.ts";
|
||||
import { appendCcDiscoveryAliases } from "../../open-sse/utils/ccDiscoveryAliases.ts";
|
||||
|
||||
const mk = (id: string, extra: Record<string, unknown> = {}) => ({
|
||||
id,
|
||||
@@ -63,6 +66,26 @@ test("non-string / empty / non-object ids never match", () => {
|
||||
assert.equal(shouldExposeClaudeEffortVariants({ id: 42 as never }), false);
|
||||
});
|
||||
|
||||
// ── isKnownClaudeEffortBaseModel ─────────────────────────────────────────────
|
||||
|
||||
test("isKnownClaudeEffortBaseModel returns true for a real effort-capable Claude model", () => {
|
||||
assert.equal(isKnownClaudeEffortBaseModel("claude-fable-5"), true);
|
||||
});
|
||||
|
||||
test("isKnownClaudeEffortBaseModel returns false for a non-Claude model", () => {
|
||||
assert.equal(isKnownClaudeEffortBaseModel("gpt-4o"), false);
|
||||
});
|
||||
|
||||
test("isKnownClaudeEffortBaseModel returns false for an unregistered model id", () => {
|
||||
assert.equal(isKnownClaudeEffortBaseModel("totally-unregistered-model-xyz"), false);
|
||||
});
|
||||
|
||||
test("isKnownClaudeEffortBaseModel returns false for a non-Claude model that also supports thinking (SC-1)", () => {
|
||||
// gpt-5.5 has supportsThinking:true in MODEL_SPECS (like 36+ other non-Claude models) —
|
||||
// the /claude/i name check is the only thing excluding it, not the thinking flag alone.
|
||||
assert.equal(isKnownClaudeEffortBaseModel("gpt-5.5"), false);
|
||||
});
|
||||
|
||||
// ── claudeEffortLevelsFor ────────────────────────────────────────────────────
|
||||
|
||||
test("xHigh is added only for models that support it", () => {
|
||||
@@ -133,3 +156,91 @@ test("never generates variants-of-variants when the list already contains effort
|
||||
.filter((id) => /-(low|medium|high|xhigh)-(low|medium|high|xhigh)$/.test(id));
|
||||
assert.deepEqual(doubleSuffixed, []);
|
||||
});
|
||||
|
||||
// ── cross-module drift guard: CLAUDE_EFFORT_SUFFIX_RE parity ────────────────
|
||||
//
|
||||
// `CLAUDE_EFFORT_SUFFIX_RE` (`/-(?:xhigh|high|medium|low)$/i`) is intentionally
|
||||
// duplicated as a local, non-exported constant in THREE sibling modules: this
|
||||
// file's module (claudeEffortVariants.ts), noThinkingAlias.ts, and
|
||||
// ccDiscoveryAliases.ts. A cross-import consolidation of that constant was
|
||||
// already proposed and explicitly reverted earlier in this project's review
|
||||
// cycle — the plan deliberately kept local duplication for these three
|
||||
// sibling modules (accepted by the Reduction Analyst). This test does NOT
|
||||
// argue for reversing that decision and must NOT be read as one. Its only
|
||||
// purpose is a behavioral drift guard: if a future edit changes the effort
|
||||
// levels recognized by one copy (e.g. adds a new level, or narrows/widens the
|
||||
// suffix pattern) without updating the other two, this test fails instead of
|
||||
// the three modules silently disagreeing about which ids carry an
|
||||
// effort-level suffix.
|
||||
test("CLAUDE_EFFORT_SUFFIX_RE stays in sync across claudeEffortVariants/noThinkingAlias/ccDiscoveryAliases (drift guard — do not consolidate, see comment above)", () => {
|
||||
// Real, registered, thinking-capable Claude model that does NOT reject
|
||||
// `thinking:{type:"disabled"}` — satisfies every module's registry-lookup
|
||||
// gate identically, so any behavioral difference below is attributable only
|
||||
// to the effort-suffix regex, not to some other per-module gating rule.
|
||||
const BASE = "claude-opus-4-5";
|
||||
const EFFORT_SUFFIXES = ["-low", "-medium", "-high", "-xhigh", "-XHIGH"];
|
||||
// Trailing tokens that look suffix-like but must NOT match the regex
|
||||
// (anchored to exactly low/medium/high/xhigh at end-of-string).
|
||||
const NON_MATCHING_SUFFIXES = ["-max", "-highest"];
|
||||
|
||||
for (const suffix of EFFORT_SUFFIXES) {
|
||||
const qualifiedId = `claude/${BASE}${suffix}`;
|
||||
assert.equal(
|
||||
shouldExposeClaudeEffortVariants(mk(qualifiedId)),
|
||||
false,
|
||||
`claudeEffortVariants must exclude ${qualifiedId}`
|
||||
);
|
||||
assert.equal(
|
||||
shouldExposeNoThinkingAlias(mk(qualifiedId)),
|
||||
false,
|
||||
`noThinkingAlias must exclude ${qualifiedId}`
|
||||
);
|
||||
const mirrored = appendCcDiscoveryAliases(
|
||||
[{ id: `cc/${BASE}${suffix}`, owned_by: "cc" }],
|
||||
() => true
|
||||
);
|
||||
assert.equal(
|
||||
mirrored.length,
|
||||
1,
|
||||
`ccDiscoveryAliases must never mirror an effort-suffixed id (${suffix})`
|
||||
);
|
||||
}
|
||||
|
||||
// Control: the identical base model WITHOUT a suffix must pass all three
|
||||
// gates — proves the suffix itself (not something else about the id) is
|
||||
// what excluded the cases above.
|
||||
assert.equal(shouldExposeClaudeEffortVariants(mk(`claude/${BASE}`)), true);
|
||||
assert.equal(shouldExposeNoThinkingAlias(mk(`claude/${BASE}`)), true);
|
||||
const baseMirror = appendCcDiscoveryAliases([{ id: `cc/${BASE}`, owned_by: "cc" }], () => true);
|
||||
assert.equal(baseMirror.length, 2, "unsuffixed id must still be mirrored");
|
||||
|
||||
// Suffix-like-but-non-matching trailing tokens must NOT be excluded by the
|
||||
// regex. This isolates the regex's specificity (exactly xhigh/high/medium/low)
|
||||
// from the models-registry prefix-matching gate: `getCanonicalModelSpecId`
|
||||
// resolves "claude-opus-4-5-max" back to the "claude-opus-4-5" spec via its
|
||||
// prefix-match fallback, so `shouldExposeClaudeEffortVariants` /
|
||||
// `shouldExposeNoThinkingAlias` still pass their registry-lookup gate here —
|
||||
// any exclusion left could only come from the suffix regex, and there is none.
|
||||
for (const suffix of NON_MATCHING_SUFFIXES) {
|
||||
const qualifiedId = `claude/${BASE}${suffix}`;
|
||||
assert.equal(
|
||||
shouldExposeClaudeEffortVariants(mk(qualifiedId)),
|
||||
true,
|
||||
`claudeEffortVariants must not treat "${suffix}" as an effort suffix`
|
||||
);
|
||||
assert.equal(
|
||||
shouldExposeNoThinkingAlias(mk(qualifiedId)),
|
||||
true,
|
||||
`noThinkingAlias must not treat "${suffix}" as an effort suffix`
|
||||
);
|
||||
const mirrored = appendCcDiscoveryAliases(
|
||||
[{ id: `cc/${BASE}${suffix}`, owned_by: "cc" }],
|
||||
() => true
|
||||
);
|
||||
assert.equal(
|
||||
mirrored.length,
|
||||
2,
|
||||
`ccDiscoveryAliases must still mirror a non-effort-suffix-looking id ("${suffix}")`
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -90,11 +90,15 @@ test("VertexExecutor.buildUrl routes partner and org-prefixed models to the glob
|
||||
);
|
||||
});
|
||||
|
||||
test("VertexExecutor.buildUrl routes current-generation Claude models to the global partner endpoint (#1985)", () => {
|
||||
test("VertexExecutor.buildUrl routes current-generation Claude models to the native Anthropic rawPredict endpoint (#1985)", () => {
|
||||
const executor = new VertexExecutor();
|
||||
|
||||
// These model IDs post-date the old pinned "claude-3-5-sonnet" / "claude-3-opus" /
|
||||
// "claude-3-haiku" prefixes and were previously misrouted to the Google-publisher path.
|
||||
// "claude-3-haiku" prefixes and were previously misrouted to the Google-publisher path,
|
||||
// then (once generalized to a "claude-" prefix, #1985) to the generic OpenAI-compatible
|
||||
// partner endpoint. Claude models use Vertex's native Anthropic Messages API
|
||||
// (publishers/anthropic/.../rawPredict) instead — the partner endpoint 404s/"malformed
|
||||
// argument"s for Claude on at least some projects.
|
||||
const claude4Sonnet = executor.buildUrl("claude-sonnet-4-6", false, 0, {
|
||||
apiKey: createServiceAccountJson({ projectId: "proj-claude" }),
|
||||
});
|
||||
@@ -104,11 +108,11 @@ test("VertexExecutor.buildUrl routes current-generation Claude models to the glo
|
||||
|
||||
assert.equal(
|
||||
claude4Sonnet,
|
||||
"https://aiplatform.googleapis.com/v1/projects/proj-claude/locations/global/endpoints/openapi/chat/completions"
|
||||
"https://aiplatform.googleapis.com/v1/projects/proj-claude/locations/us-central1/publishers/anthropic/models/claude-sonnet-4-6:rawPredict"
|
||||
);
|
||||
assert.equal(
|
||||
claude4Haiku,
|
||||
"https://aiplatform.googleapis.com/v1/projects/proj-claude/locations/global/endpoints/openapi/chat/completions"
|
||||
"https://aiplatform.googleapis.com/v1/projects/proj-claude/locations/us-central1/publishers/anthropic/models/claude-haiku-4-5@20251001:rawPredict"
|
||||
);
|
||||
});
|
||||
|
||||
@@ -238,3 +242,110 @@ test("VertexExecutor.execute rejects incomplete Service Account JSON clearly", a
|
||||
/missing required fields/
|
||||
);
|
||||
});
|
||||
|
||||
test("VertexExecutor.execute strips the client's model field and injects anthropic_version for Claude models", async () => {
|
||||
const executor = new VertexExecutor();
|
||||
const originalFetch = globalThis.fetch;
|
||||
const calls = [];
|
||||
|
||||
globalThis.fetch = async (url, options) => {
|
||||
calls.push({ url: String(url), body: String(options?.body || "") });
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
id: "msg_1",
|
||||
type: "message",
|
||||
role: "assistant",
|
||||
model: "claude-sonnet-4-6",
|
||||
content: [{ type: "text", text: "hi" }],
|
||||
stop_reason: "end_turn",
|
||||
usage: { input_tokens: 3, output_tokens: 1 },
|
||||
}),
|
||||
{ status: 200, headers: { "Content-Type": "application/json" } }
|
||||
);
|
||||
};
|
||||
|
||||
try {
|
||||
await executor.execute({
|
||||
model: "claude-sonnet-4-6",
|
||||
// rawPredict rejects a body-level "model" field ("Extra inputs are not permitted") since
|
||||
// the model is already encoded in the URL — the openai→claude request translator copies
|
||||
// the client's model field over, so the executor must strip it before sending.
|
||||
body: { model: "vertex/claude-sonnet-4-6", messages: [{ role: "user", content: "hi" }] },
|
||||
stream: false,
|
||||
credentials: {
|
||||
apiKey: createServiceAccountJson({ projectId: "proj-claude" }),
|
||||
accessToken: "ya29.claude",
|
||||
},
|
||||
});
|
||||
|
||||
assert.equal(calls.length, 1);
|
||||
const sentBody = JSON.parse(calls[0].body);
|
||||
assert.equal(sentBody.model, undefined);
|
||||
assert.equal(sentBody.anthropic_version, "vertex-2023-10-16");
|
||||
assert.deepEqual(sentBody.messages, [{ role: "user", content: "hi" }]);
|
||||
} finally {
|
||||
globalThis.fetch = originalFetch;
|
||||
}
|
||||
});
|
||||
|
||||
test("VertexExecutor.execute synthesizes a genuine Anthropic-format SSE stream when rawPredict returns a complete JSON body for a streaming request", async () => {
|
||||
const executor = new VertexExecutor();
|
||||
const originalFetch = globalThis.fetch;
|
||||
|
||||
// rawPredict is a non-streaming endpoint — Vertex can still hand back a complete,
|
||||
// non-chunked JSON body for a request that asked for stream:true. Without synthesis
|
||||
// this reaches the client as a single JSON blob the OpenAI-only jsonToSse fallback
|
||||
// can't parse (it looks for "choices", not Anthropic's "content" shape), producing
|
||||
// "Provider returned empty content" instead of real streamed text.
|
||||
globalThis.fetch = async () =>
|
||||
new Response(
|
||||
JSON.stringify({
|
||||
id: "msg_stream_1",
|
||||
type: "message",
|
||||
role: "assistant",
|
||||
model: "claude-sonnet-4-6",
|
||||
content: [{ type: "text", text: "hello" }],
|
||||
stop_reason: "end_turn",
|
||||
stop_sequence: null,
|
||||
usage: { input_tokens: 5, output_tokens: 2 },
|
||||
}),
|
||||
{ status: 200, headers: { "Content-Type": "application/json" } }
|
||||
);
|
||||
|
||||
try {
|
||||
const result = await executor.execute({
|
||||
model: "claude-sonnet-4-6",
|
||||
body: { messages: [{ role: "user", content: "hi" }] },
|
||||
stream: true,
|
||||
credentials: {
|
||||
apiKey: createServiceAccountJson({ projectId: "proj-claude" }),
|
||||
accessToken: "ya29.claude",
|
||||
},
|
||||
});
|
||||
|
||||
const response = result.response;
|
||||
assert.equal(response.status, 200);
|
||||
assert.equal(response.headers.get("content-type"), "text/event-stream");
|
||||
|
||||
const text = await response.text();
|
||||
assert.match(text, /event: message_start/);
|
||||
assert.match(text, /"type":"content_block_delta".*"text":"hello"/);
|
||||
assert.match(text, /event: message_stop/);
|
||||
|
||||
const dataLines = text
|
||||
.split(/\r?\n/)
|
||||
.filter((line) => line.startsWith("data:"))
|
||||
.map((line) => JSON.parse(line.slice(5).trim()));
|
||||
const types = dataLines.map((d) => d.type);
|
||||
assert.deepEqual(types, [
|
||||
"message_start",
|
||||
"content_block_start",
|
||||
"content_block_delta",
|
||||
"content_block_stop",
|
||||
"message_delta",
|
||||
"message_stop",
|
||||
]);
|
||||
} finally {
|
||||
globalThis.fetch = originalFetch;
|
||||
}
|
||||
});
|
||||
|
||||
@@ -73,7 +73,11 @@ test("applyNoThinkingAlias expresses reasoning_effort:none without a thinking bl
|
||||
// #6879: a thinks-by-default OpenAI-shape model must carry reasoning_effort:"none"
|
||||
// explicitly (not merely have the field deleted), so suppression actually takes
|
||||
// effect downstream; the Responses-shaped `reasoning` object is still dropped.
|
||||
assert.equal(body.reasoning_effort, "none", "reasoning_effort must express none, not be stripped");
|
||||
assert.equal(
|
||||
body.reasoning_effort,
|
||||
"none",
|
||||
"reasoning_effort must express none, not be stripped"
|
||||
);
|
||||
assert.ok(!("reasoning" in body), "reasoning object must be dropped");
|
||||
});
|
||||
|
||||
@@ -96,11 +100,7 @@ test("applyNoThinkingAlias ignores a malformed prefix-only model", () => {
|
||||
const body: Record<string, unknown> = { model: "no-think/" };
|
||||
const res = applyNoThinkingAlias(body, { claudeFormat: true });
|
||||
assert.equal(res.applied, false);
|
||||
assert.equal(
|
||||
body.model,
|
||||
"no-think/",
|
||||
"left untouched when nothing follows the prefix"
|
||||
);
|
||||
assert.equal(body.model, "no-think/", "left untouched when nothing follows the prefix");
|
||||
});
|
||||
|
||||
// ── catalog gating ───────────────────────────────────────────────────────────
|
||||
@@ -120,28 +120,16 @@ test("shouldExposeNoThinkingAlias rejects models where suppression is meaningles
|
||||
// combos are virtual, never aliased
|
||||
assert.equal(shouldExposeNoThinkingAlias(entry("my-combo", "combo")), false);
|
||||
// never double-alias
|
||||
assert.equal(
|
||||
shouldExposeNoThinkingAlias(entry("no-think/anthropic/claude-opus-4-5")),
|
||||
false
|
||||
);
|
||||
assert.equal(shouldExposeNoThinkingAlias(entry("no-think/anthropic/claude-opus-4-5")), false);
|
||||
});
|
||||
|
||||
test("appendNoThinkingVariants adds one variant per eligible model and preserves the rest", () => {
|
||||
const models = [entry("claude-opus-4-5"), entry("gpt-4o", "openai"), entry("claude-fable-5")];
|
||||
const out = appendNoThinkingVariants(models);
|
||||
const ids = out.map((m) => m.id);
|
||||
assert.ok(
|
||||
ids.includes("no-think/claude-opus-4-5"),
|
||||
"eligible model gets a variant"
|
||||
);
|
||||
assert.ok(
|
||||
!ids.includes("no-think/gpt-4o"),
|
||||
"non-thinking model has no variant"
|
||||
);
|
||||
assert.ok(
|
||||
!ids.includes("no-think/claude-fable-5"),
|
||||
"reject-disabled model has no variant"
|
||||
);
|
||||
assert.ok(ids.includes("no-think/claude-opus-4-5"), "eligible model gets a variant");
|
||||
assert.ok(!ids.includes("no-think/gpt-4o"), "non-thinking model has no variant");
|
||||
assert.ok(!ids.includes("no-think/claude-fable-5"), "reject-disabled model has no variant");
|
||||
assert.equal(out.length, models.length + 1, "exactly one variant appended");
|
||||
// originals preserved up front
|
||||
assert.deepEqual(out.slice(0, 3), models);
|
||||
@@ -157,22 +145,42 @@ test("appendNoThinkingVariants normalizes alias prefix to canonical when aliasTo
|
||||
const aliasToCanonical = { cc: "claude" };
|
||||
const out = appendNoThinkingVariants(models, aliasToCanonical);
|
||||
const ids = out.map((m) => m.id);
|
||||
assert.ok(
|
||||
ids.includes("no-think/claude/claude-opus-4-5"),
|
||||
"uses canonical prefix"
|
||||
);
|
||||
assert.ok(
|
||||
!ids.includes("no-think/cc/claude-opus-4-5"),
|
||||
"alias prefix not used"
|
||||
);
|
||||
assert.ok(ids.includes("no-think/claude/claude-opus-4-5"), "uses canonical prefix");
|
||||
assert.ok(!ids.includes("no-think/cc/claude-opus-4-5"), "alias prefix not used");
|
||||
});
|
||||
|
||||
test("appendNoThinkingVariants keeps alias prefix when no map is provided", () => {
|
||||
const models = [entry("cc/claude-opus-4-5")];
|
||||
const out = appendNoThinkingVariants(models);
|
||||
const ids = out.map((m) => m.id);
|
||||
assert.ok(ids.includes("no-think/cc/claude-opus-4-5"), "alias prefix preserved");
|
||||
});
|
||||
|
||||
test("appendNoThinkingVariants keeps root bare even when id carries a provider prefix", () => {
|
||||
const models = [entry("vertex/claude-opus-4-5", "vertex")];
|
||||
const out = appendNoThinkingVariants(models);
|
||||
const variant = out.find((m) => m.id === "no-think/vertex/claude-opus-4-5");
|
||||
assert.ok(variant, "variant with the fully-qualified id must exist");
|
||||
assert.equal(
|
||||
variant!.root,
|
||||
"no-think/claude-opus-4-5",
|
||||
"root must be bare (no embedded provider segment), matching the effort-variant convention"
|
||||
);
|
||||
});
|
||||
|
||||
test("shouldExposeNoThinkingAlias rejects an already effort-suffixed id", () => {
|
||||
assert.equal(shouldExposeNoThinkingAlias(entry("vertex/claude-sonnet-5-high")), false);
|
||||
assert.equal(shouldExposeNoThinkingAlias(entry("claude-opus-4-5-xhigh")), false);
|
||||
});
|
||||
|
||||
test("appendNoThinkingVariants does not synthesize a no-think variant of an effort variant", () => {
|
||||
// Simulates the real pipeline order in catalogResponse.ts: appendClaudeEffortVariants
|
||||
// runs first and produces an id like this before appendNoThinkingVariants ever sees it.
|
||||
const models = [entry("vertex/claude-sonnet-5-high")];
|
||||
const out = appendNoThinkingVariants(models);
|
||||
assert.equal(out, models, "no variant should be added for an effort-suffixed id");
|
||||
assert.ok(
|
||||
ids.includes("no-think/cc/claude-opus-4-5"),
|
||||
"alias prefix preserved"
|
||||
!out.some((m) => m.id === "no-think/vertex/claude-sonnet-5-high"),
|
||||
"the incoherent combined id must never be advertised"
|
||||
);
|
||||
});
|
||||
|
||||
@@ -37,3 +37,37 @@ test("OpenCode Free playground uses its routing alias instead of the reserved pr
|
||||
assert.equal(getProviderAlias("opencode"), "oc");
|
||||
assert.equal(qualifyPlaygroundModel("big-pickle", getProviderAlias("opencode")), "oc/big-pickle");
|
||||
});
|
||||
|
||||
test("qualifyPlaygroundModel inserts the provider after the no-think prefix, not before it", () => {
|
||||
assert.equal(
|
||||
qualifyPlaygroundModel("no-think/claude-sonnet-5", "vertex"),
|
||||
"no-think/vertex/claude-sonnet-5"
|
||||
);
|
||||
});
|
||||
|
||||
test("qualifyPlaygroundModel does not double-qualify an already-qualified no-think id", () => {
|
||||
assert.equal(
|
||||
qualifyPlaygroundModel("no-think/vertex/claude-sonnet-5", "vertex"),
|
||||
"no-think/vertex/claude-sonnet-5"
|
||||
);
|
||||
});
|
||||
|
||||
test("qualifyPlaygroundModel does not mistake a provider-name-prefix collision for already-qualified", () => {
|
||||
// routingPrefix "vertex" must not match "vertex-eu/..." as already-qualified just because
|
||||
// it starts with the same characters — the check requires an exact "vertex/" segment
|
||||
// boundary. A naive `inner.startsWith(routingPrefix)` (no slash) would wrongly skip
|
||||
// qualification here and leave the provider segment un-inserted.
|
||||
assert.equal(
|
||||
qualifyPlaygroundModel("no-think/vertex-eu/claude-sonnet-5", "vertex"),
|
||||
"no-think/vertex/vertex-eu/claude-sonnet-5"
|
||||
);
|
||||
});
|
||||
|
||||
test("LlmChatCard's local NO_THINKING_PREFIX literal matches the canonical constant", async () => {
|
||||
// Drift guard: LlmChatCard.tsx deliberately hardcodes "no-think/" as a literal instead
|
||||
// of importing NO_THINKING_PREFIX from open-sse/utils/noThinkingAlias.ts (avoids pulling
|
||||
// server-side catalog modules into the client bundle — see Step 1). This test file is not
|
||||
// client-bundled, so it can safely import the real constant and assert they never drift.
|
||||
const { NO_THINKING_PREFIX } = await import("../../open-sse/utils/noThinkingAlias.ts");
|
||||
assert.equal(NO_THINKING_PREFIX, "no-think/");
|
||||
});
|
||||
|
||||
292
tests/unit/vertex-passthrough-model-lockout.test.ts
Normal file
292
tests/unit/vertex-passthrough-model-lockout.test.ts
Normal file
@@ -0,0 +1,292 @@
|
||||
// Regression guard: after adding passthroughModels: true to Vertex's registry entry, a 404 on
|
||||
// one Vertex model (e.g. a stale/synthetic model id) must lock out only that model, not cool
|
||||
// down the whole connection — mirrors the existing ollama-cloud/bedrock protection. Before this
|
||||
// fix, hasPerModelQuota("vertex", ...) was false, so any 404 on Vertex cooled the whole
|
||||
// connection for COOLDOWN_MS.notFound (2 minutes), per errorConfig.ts's generic status_404 rule.
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-vertex-404-"));
|
||||
process.env.DATA_DIR = TEST_DATA_DIR;
|
||||
|
||||
const core = await import("../../src/lib/db/core.ts");
|
||||
const providersDb = await import("../../src/lib/db/providers.ts");
|
||||
const auth = await import("../../src/sse/services/auth.ts");
|
||||
const accountFallback = await import("../../open-sse/services/accountFallback.ts");
|
||||
|
||||
async function resetStorage() {
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
fs.mkdirSync(TEST_DATA_DIR, { recursive: true });
|
||||
}
|
||||
|
||||
async function seedVertex() {
|
||||
return providersDb.createProviderConnection({
|
||||
provider: "vertex",
|
||||
authType: "apikey",
|
||||
apiKey: "vertex-key",
|
||||
isActive: true,
|
||||
testStatus: "active",
|
||||
});
|
||||
}
|
||||
|
||||
test.after(() => {
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test('hasPerModelQuota("vertex", ...) is true after the passthroughModels registry flag', () => {
|
||||
assert.equal(accountFallback.hasPerModelQuota("vertex", "claude-sonnet-5"), true);
|
||||
});
|
||||
|
||||
test("404 on one Vertex model locks only that model, connection stays active", async () => {
|
||||
await resetStorage();
|
||||
const conn = await seedVertex();
|
||||
|
||||
const result = await auth.markAccountUnavailable(
|
||||
conn.id,
|
||||
404,
|
||||
"model not found",
|
||||
"vertex",
|
||||
"claude-sonnet-5-high"
|
||||
);
|
||||
assert.equal(result.shouldFallback, true);
|
||||
|
||||
const after = await providersDb.getProviderConnectionById(conn.id);
|
||||
assert.equal(after.testStatus, "active");
|
||||
assert.ok(!after.rateLimitedUntil, "connection must not be rate-limited");
|
||||
|
||||
const lockout = accountFallback.getModelLockoutInfo("vertex", conn.id, "claude-sonnet-5-high");
|
||||
assert.equal(lockout?.reason, "not_found");
|
||||
|
||||
// A sibling model on the same connection must remain immediately eligible.
|
||||
const sibling = accountFallback.getModelLockoutInfo("vertex", conn.id, "gemini-3.1-pro-preview");
|
||||
assert.equal(sibling, null);
|
||||
});
|
||||
|
||||
test("503 on one Vertex model locks only that model, connection stays active (proves the fix isn't 404-specific)", async () => {
|
||||
// hasPerModelQuota's gate covers status === 404 || status === 429 || status >= 500
|
||||
// (auth.ts:2024) in one shared branch — 502/503/504 keep the model-lockout path (only
|
||||
// the exact 500 is exempted per #5976). This mirrors the 404 test above with a 5xx to
|
||||
// confirm passthroughModels doesn't just fix the specific 404 symptom reported.
|
||||
await resetStorage();
|
||||
const conn = await seedVertex();
|
||||
|
||||
const result = await auth.markAccountUnavailable(
|
||||
conn.id,
|
||||
503,
|
||||
"Service Unavailable",
|
||||
"vertex",
|
||||
"claude-sonnet-5-high"
|
||||
);
|
||||
assert.equal(result.shouldFallback, true);
|
||||
|
||||
const after = await providersDb.getProviderConnectionById(conn.id);
|
||||
assert.equal(after.testStatus, "active");
|
||||
assert.ok(!after.rateLimitedUntil, "connection must not be rate-limited");
|
||||
|
||||
const lockout = accountFallback.getModelLockoutInfo("vertex", conn.id, "claude-sonnet-5-high");
|
||||
assert.equal(lockout?.reason, "server_error");
|
||||
|
||||
const sibling = accountFallback.getModelLockoutInfo("vertex", conn.id, "gemini-3.1-pro-preview");
|
||||
assert.equal(sibling, null);
|
||||
});
|
||||
|
||||
test("403 PERMISSION_DENIED on Vertex locks only that model too (accepted trade-off, see plan)", async () => {
|
||||
await resetStorage();
|
||||
const conn = await seedVertex();
|
||||
|
||||
// Google Cloud uses the literal "PERMISSION_DENIED" status name for BOTH a
|
||||
// model-specific denial and a connection-wide IAM/API-disabled failure — this
|
||||
// fix cannot distinguish them (no live Vertex credential test in this plan), so
|
||||
// it intentionally treats both as a per-model lockout post-passthroughModels.
|
||||
const result = await auth.markAccountUnavailable(
|
||||
conn.id,
|
||||
403,
|
||||
"PERMISSION_DENIED: the caller does not have permission",
|
||||
"vertex",
|
||||
"claude-sonnet-5-high"
|
||||
);
|
||||
assert.equal(result.shouldFallback, true);
|
||||
|
||||
const after = await providersDb.getProviderConnectionById(conn.id);
|
||||
assert.equal(after.testStatus, "active");
|
||||
assert.ok(!after.rateLimitedUntil, "connection must not be rate-limited");
|
||||
|
||||
const lockout = accountFallback.getModelLockoutInfo("vertex", conn.id, "claude-sonnet-5-high");
|
||||
assert.equal(lockout?.reason, "forbidden");
|
||||
});
|
||||
|
||||
// ── 403 disambiguation via google.rpc.ErrorInfo (Task 7) ────────────────────
|
||||
// Google Cloud's documented error format (https://cloud.google.com/apis/design/errors#error_info)
|
||||
// lets us tell a genuinely connection-wide PERMISSION_DENIED (API disabled, or a
|
||||
// project-level IAM denial) apart from one scoped to a single model — the former must
|
||||
// fall through to the existing connection-wide cooldown instead of the #3027 per-model
|
||||
// lockout path, since a per-model lockout would leave a broken connection "active".
|
||||
|
||||
test("403 with SERVICE_DISABLED ErrorInfo reason cools down the whole Vertex connection", async () => {
|
||||
await resetStorage();
|
||||
const conn = await seedVertex();
|
||||
|
||||
const body = JSON.stringify({
|
||||
error: {
|
||||
status: "PERMISSION_DENIED",
|
||||
details: [
|
||||
{
|
||||
"@type": "type.googleapis.com/google.rpc.ErrorInfo",
|
||||
reason: "SERVICE_DISABLED",
|
||||
domain: "googleapis.com",
|
||||
metadata: { service: "aiplatform.googleapis.com", consumer: "projects/12345" },
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
const result = await auth.markAccountUnavailable(
|
||||
conn.id,
|
||||
403,
|
||||
body,
|
||||
"vertex",
|
||||
"claude-sonnet-5-high"
|
||||
);
|
||||
assert.equal(result.shouldFallback, true);
|
||||
|
||||
const after = await providersDb.getProviderConnectionById(conn.id);
|
||||
assert.equal(after.testStatus, "unavailable");
|
||||
assert.ok(after.rateLimitedUntil, "connection must be cooled down, not left active");
|
||||
|
||||
// The #3027 per-model lockout path must NOT have run.
|
||||
const lockout = accountFallback.getModelLockoutInfo("vertex", conn.id, "claude-sonnet-5-high");
|
||||
assert.equal(lockout, null);
|
||||
});
|
||||
|
||||
test("403 with IAM_PERMISSION_DENIED reason and a model-scoped resource still locks only that model", async () => {
|
||||
await resetStorage();
|
||||
const conn = await seedVertex();
|
||||
|
||||
const body = JSON.stringify({
|
||||
error: {
|
||||
status: "PERMISSION_DENIED",
|
||||
details: [
|
||||
{
|
||||
"@type": "type.googleapis.com/google.rpc.ErrorInfo",
|
||||
reason: "IAM_PERMISSION_DENIED",
|
||||
domain: "iam.googleapis.com",
|
||||
metadata: {
|
||||
permission: "aiplatform.endpoints.predict",
|
||||
resource:
|
||||
"projects/12345/locations/us-central1/publishers/google/models/claude-sonnet-5",
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
const result = await auth.markAccountUnavailable(
|
||||
conn.id,
|
||||
403,
|
||||
body,
|
||||
"vertex",
|
||||
"claude-sonnet-5-high"
|
||||
);
|
||||
assert.equal(result.shouldFallback, true);
|
||||
|
||||
const after = await providersDb.getProviderConnectionById(conn.id);
|
||||
assert.equal(after.testStatus, "active");
|
||||
assert.ok(!after.rateLimitedUntil, "connection must not be rate-limited");
|
||||
|
||||
const lockout = accountFallback.getModelLockoutInfo("vertex", conn.id, "claude-sonnet-5-high");
|
||||
assert.equal(lockout?.reason, "forbidden");
|
||||
});
|
||||
|
||||
test("403 with IAM_PERMISSION_DENIED reason and a project-level resource cools down the whole connection", async () => {
|
||||
await resetStorage();
|
||||
const conn = await seedVertex();
|
||||
|
||||
const body = JSON.stringify({
|
||||
error: {
|
||||
status: "PERMISSION_DENIED",
|
||||
details: [
|
||||
{
|
||||
"@type": "type.googleapis.com/google.rpc.ErrorInfo",
|
||||
reason: "IAM_PERMISSION_DENIED",
|
||||
domain: "iam.googleapis.com",
|
||||
metadata: {
|
||||
permission: "aiplatform.googleapis.com/models.predict",
|
||||
resource: "projects/12345",
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
const result = await auth.markAccountUnavailable(
|
||||
conn.id,
|
||||
403,
|
||||
body,
|
||||
"vertex",
|
||||
"claude-sonnet-5-high"
|
||||
);
|
||||
assert.equal(result.shouldFallback, true);
|
||||
|
||||
const after = await providersDb.getProviderConnectionById(conn.id);
|
||||
assert.equal(after.testStatus, "unavailable");
|
||||
assert.ok(after.rateLimitedUntil, "connection must be cooled down, not left active");
|
||||
|
||||
const lockout = accountFallback.getModelLockoutInfo("vertex", conn.id, "claude-sonnet-5-high");
|
||||
assert.equal(lockout, null);
|
||||
});
|
||||
|
||||
test("multi-detail error body correlates reason+resource per-detail, not across details", async () => {
|
||||
// Adversarial case: detail[0] has a model-scoped resource under an unrelated reason,
|
||||
// detail[1] carries the actual IAM_PERMISSION_DENIED with a project-level resource. A
|
||||
// naive independent-regex scan would match detail[0]'s resource against detail[1]'s
|
||||
// reason and wrongly conclude "model-scoped" — this must resolve to connection-wide.
|
||||
await resetStorage();
|
||||
const conn = await seedVertex();
|
||||
|
||||
const body = JSON.stringify({
|
||||
error: {
|
||||
status: "PERMISSION_DENIED",
|
||||
details: [
|
||||
{
|
||||
"@type": "type.googleapis.com/google.rpc.ErrorInfo",
|
||||
reason: "SOME_OTHER_REASON",
|
||||
domain: "iam.googleapis.com",
|
||||
metadata: {
|
||||
resource:
|
||||
"projects/12345/locations/us-central1/publishers/google/models/claude-sonnet-5",
|
||||
},
|
||||
},
|
||||
{
|
||||
"@type": "type.googleapis.com/google.rpc.ErrorInfo",
|
||||
reason: "IAM_PERMISSION_DENIED",
|
||||
domain: "iam.googleapis.com",
|
||||
metadata: {
|
||||
permission: "aiplatform.googleapis.com/models.predict",
|
||||
resource: "projects/12345",
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
const result = await auth.markAccountUnavailable(
|
||||
conn.id,
|
||||
403,
|
||||
body,
|
||||
"vertex",
|
||||
"claude-sonnet-5-high"
|
||||
);
|
||||
assert.equal(result.shouldFallback, true);
|
||||
|
||||
const after2 = await providersDb.getProviderConnectionById(conn.id);
|
||||
assert.equal(after2.testStatus, "unavailable");
|
||||
assert.ok(after2.rateLimitedUntil, "connection must be cooled down, not left active");
|
||||
|
||||
const lockout2 = accountFallback.getModelLockoutInfo("vertex", conn.id, "claude-sonnet-5-high");
|
||||
assert.equal(lockout2, null);
|
||||
});
|
||||
Reference in New Issue
Block a user