Merge branch 'pr/9006' into feat/personal-build

This commit is contained in:
Egor
2026-07-31 08:39:36 +03:00
24 changed files with 1103 additions and 84 deletions

View File

@@ -0,0 +1 @@
- **fix(executors):** Vertex AI now routes Claude models through the native Anthropic `rawPredict` endpoint instead of the generic OpenAI-compatible partner endpoint, and synthesizes a real streaming response so Claude-via-Vertex works with `stream: true` ([#8909](https://github.com/diegosouzapw/OmniRoute/pull/8909)) — thanks @wgordon17

View File

@@ -0,0 +1,14 @@
- **fix(sse):** Claude reasoning-effort suffix ids (`-high`/`-low`/`-medium`/`-xhigh`) now strip
correctly on any provider serving a real Claude model, not just the direct Anthropic provider
([#9006](https://github.com/diegosouzapw/OmniRoute/pull/9006))
- **fix(sse):** the no-thinking (`no-think/`) catalog variant's provider-qualification bug — which
made it unusable outside the direct provider, both in the discovery catalog and the dashboard
playground — is fixed ([#9006](https://github.com/diegosouzapw/OmniRoute/pull/9006))
- **fix(sse):** a single unrecognized model id on a Vertex connection no longer cools down every
other model on that connection for 2 minutes — Vertex 404s are now scoped to a per-model
lockout via `passthroughModels` instead of a connection-wide cooldown
([#9006](https://github.com/diegosouzapw/OmniRoute/pull/9006))
- **fix(sse):** Vertex `PERMISSION_DENIED` 403s are now disambiguated using Google's own
documented error format — a genuinely connection-wide cause (API disabled, project-level IAM
denial) still cools the whole connection, while a model-specific denial locks out only that
model ([#9006](https://github.com/diegosouzapw/OmniRoute/pull/9006))

View File

@@ -1,4 +1,5 @@
{
"_rebaseline_2026_07_30_9006_vertex_claude_catalog_dispatch": "PR #9006 (fix/vertex-claude-catalog-dispatch): three files, two causes. (1) src/sse/handlers/chat.ts 1845->1846 (+1): NOT this PR's own growth — this PR never touches chat.ts at all. Measured 1846 (split(\"\\n\").length) at this PR's own merge-base (before any of its 11 commits), so the drift was already inherited from already-merged PRs on release/v3.8.50 (fast-gates PR->release do not run check:file-size, same root cause as _rebaseline_2026_07_25_v3849_basered_filesize and _rebaseline_2026_07_02_5798_release_green) — no offending branch left to fix. (2) src/sse/services/auth.ts 2508->2512 (+4 net, after extraction — see below) and open-sse/handlers/chatCore.ts 5020->5023 (+3, comment-only): genuine own growth. auth.ts adds Vertex 403 PERMISSION_DENIED disambiguation (Google's google.rpc.ErrorInfo proto distinguishes a connection-wide cause — SERVICE_DISABLED, or IAM_PERMISSION_DENIED against a project-level resource — from a model-specific one scoped to a .../models/<id> resource), added mid-PR after a quality-gate reviewer flagged the plan's originally-accepted \"Vertex 403 always -> per-model lockout\" trade-off. The actual classification logic (~40 lines) was EXTRACTED into a new leaf module src/sse/services/vertexErrorClassifier.ts (mirrors the googApiKeyAuth.ts precedent, _rebaseline_2026_07_14_7034_goog_api_key), leaving only the irreducible call-site wiring in the frozen file: a 1-line import plus widening the existing #3027 per-model-403 guard condition. chatCore.ts's +3 is a pure comment expansion (no functional change) clarifying that the adjacent effort-suffix strip is no longer unconditional for every provider, requested by a separate quality-gate code-reviewer finding; not extractable (it's a comment). Auth.ts's disambiguation logic covered by 3 new test cases in tests/unit/vertex-passthrough-model-lockout.test.ts (SERVICE_DISABLED, IAM_PERMISSION_DENIED+model-resource, IAM_PERMISSION_DENIED+project-resource) plus a 4th regression test for a multi-detail-body correlation bug (reason and resource must be read from the SAME ErrorInfo detail, not independently regexed across the whole body) found by an adversarial quality-gate pass and fixed before merge.",
"_rebaseline_2026_07_24_8470_hyperagent_sticky_thread": "PR #8470 (artickc, fix/hyperagent-tool-loop-thread-sticky) own growth: open-sse/executors/hyperagent.ts 936->1025 (wc -l; check-file-size.mjs counts via split(\"\\n\").length so the gate sees 937->1026, +89, crosses the 1000 cap). Fixes a real bug where a reverse-conversion proxy (text-Intent/JSON to Claude Code native tool_calls) rewrites assistant messages between agentic tool-loop turns, breaking HyperAgent's conversation-prefix fingerprint and cold-starting the thread mid tool-loop. Adds Anthropic tool_use/tool_result flattening to extractMessageText() plus a new rootUserFingerprint()/root-key lookup tier in resolveHyperAgentThreadBinding()/storeHyperAgentThreadAfterTurn() so the thread stays sticky across the tool loop. Cohesive additions inside the existing single-file executor; not extractable without splitting the executor mid-request-flow. Covered by tests/unit/executor-hyperagent.test.ts (19/19, +5 new cases for tool_result/tool_use flattening + root-key stickiness). Pre-merge review flagged a cross-conversation root-key collision risk (tracked in the PR's own mandatory pre-merge checklist, not yet addressed) — unrelated to this file-size ratchet, tracked separately by /fix-prs.",
"_rebaseline_2026_07_25_8494_capability_filter_fail_closed": "PR #8494 (fix/capability-filters-fail-closed, #8488) own growth: open-sse/services/combo.ts 3640->3693 (+53) adds a fail-closed guard after filterTargetsByRequestCompatibility() — when every eligible target is excluded by request-capability filtering (vision/tools/etc) instead of quota/health, the combo now returns an explicit `capability_mismatch` 400 (describeCapabilityFilterExhaustion, imported from combo/comboStructure.ts) rather than silently falling through to a generic no-targets error, plus a `compatFilterFailOpen` escape hatch (combo config OR settings) mirrored at both the main/auto and round-robin call sites for symmetry. combo/comboStructure.ts (previously under cap, un-frozen) grows 794->918 (+124) — new home for describeCapabilityFilterExhaustion + providerSupportsEmulatedToolCalling (#5240 emulated tool-calling exemption so fail-closed does not regress prompt-emulation-only combos like all-chatgpt-web). Irreducible orchestration wiring at the existing filter chokepoint (same precedent as #7301's universal-cooldown-retry generalization). Companion test tests/unit/combo-routing-engine.test.ts 3409->3449 (+40, fail-closed/fail-open coverage across both call sites) also rebaselined. Covered by tests/unit/8488-capability-filter-fail-closed.test.ts (new) + 95/95 passing across both files. Structural shrink of combo.ts tracked in #3501.",
"_rebaseline_2026_07_25_8499_ts7_result_union_predicates": "PR #8499 (backryun, chore/ts7-types-executor-scattered) own growth: muse-spark-web.ts 1396->1405 (+9, irreducible). Under this workspace's `strictNullChecks: false`, the boolean-literal discriminant on `GraphqlResult` (`{ ok: true } | { ok: false; error: string }`) narrows the positive `.ok===true` branch but leaves `!result.ok` at the full union under TS7, making `.error` unreachable to the checker at the two call sites (warmup, mode-switch). Fixed by adding a single `isGraphqlFailure()` type-predicate helper (doc comment + 3-line body) reused at both call sites instead of duplicating the predicate inline — not extractable to a shared module without splitting a single-file executor's local narrowing helper out of its own file. Covered by the existing muse-spark-web executor test suite (no behavior change, pure narrowing fix).",
@@ -349,7 +350,7 @@
"open-sse/executors/deepseek-web.ts": 1148,
"open-sse/executors/grok-web.ts": 1044,
"open-sse/executors/muse-spark-web.ts": 1405,
"open-sse/handlers/chatCore.ts": 5020,
"open-sse/handlers/chatCore.ts": 5023,
"open-sse/handlers/imageGeneration.ts": 3101,
"open-sse/handlers/responseSanitizer.ts": 1115,
"open-sse/handlers/search.ts": 1536,
@@ -400,8 +401,8 @@
"src/shared/components/RequestLoggerV2.tsx": 1629,
"src/shared/components/analytics/charts.tsx": 1035,
"src/shared/services/cliRuntime.ts": 1122,
"src/sse/handlers/chat.ts": 1845,
"src/sse/services/auth.ts": 2508,
"src/sse/handlers/chat.ts": 1846,
"src/sse/services/auth.ts": 2512,
"tests/unit/account-fallback-service.test.ts": 1572,
"tests/unit/provider-validation-specialty.test.ts": 2980,
"open-sse/executors/hyperagent.ts": 1026

View File

@@ -30,4 +30,5 @@ export const vertexProvider: RegistryEntry = {
{ id: "claude-opus-4-7", name: "Claude Opus 4.7 (Vertex)" },
{ id: "claude-sonnet-4-6", name: "Claude Sonnet 4.6 (Vertex)" },
],
passthroughModels: true,
};

View File

@@ -138,13 +138,149 @@ function isPartnerModel(model: string) {
return [...PARTNER_MODELS].some((prefix) => normalizedModel.startsWith(prefix));
}
// Anthropic models need their own branch: they use Vertex's native Anthropic Messages API
// (publishers/anthropic/.../rawPredict), not the generic OpenAI-compatible partner endpoint the
// other PARTNER_MODELS entries (DeepSeek, Qwen, Llama, Mistral, GLM) go through — the OpenAI-shaped
// endpoint 404s/"malformed argument"s for Claude models on at least some projects.
function isClaudeModel(model: string) {
return model.toLowerCase().startsWith("claude-");
}
// Defensive normalizer: target-format resolution for manually-added custom Claude models under
// "vertex"/"vertex-partner" was observed sending a Gemini-shaped body (contents/parts) to the
// Anthropic rawPredict endpoint instead of the configured "claude" format, causing a hard
// "messages: Field required" error upstream regardless of the stored per-model targetFormat. This
// converts a Gemini-shaped body to Anthropic Messages shape so the executor works either way,
// independent of that unresolved upstream resolution gap.
function toAnthropicBody(body: Record<string, unknown>): Record<string, unknown> {
const contents = body.contents as Array<{ role?: string; parts?: Array<{ text?: string }> }> | undefined;
if (!Array.isArray(contents)) return body;
const messages = contents.map((c) => ({
role: c.role === "model" ? "assistant" : "user",
content: (c.parts || []).map((p) => p.text || "").join(""),
}));
const generationConfig = body.generationConfig as { maxOutputTokens?: number } | undefined;
const systemInstruction = body.systemInstruction as { parts?: Array<{ text?: string }> } | undefined;
const converted: Record<string, unknown> = {
messages,
max_tokens: generationConfig?.maxOutputTokens || 4096,
};
if (systemInstruction?.parts?.length) {
converted.system = systemInstruction.parts.map((p) => p.text || "").join("");
}
return converted;
}
// rawPredict always returns a single complete JSON body, never real SSE framing (see buildUrl).
// When the caller actually requested a stream, synthesize a genuine Anthropic-native event
// sequence from that JSON so the existing claude-to-openai.ts (and sibling) response translators
// — which already parse real message_start/content_block_*/message_delta/message_stop events —
// can consume it correctly, instead of relying on the OpenAI-`choices`-only JSON→SSE fallback
// (open-sse/utils/jsonToSse.ts) which cannot represent Anthropic's native response shape at all.
function synthesizeClaudeSse(response: Record<string, unknown>): string {
const messageId = typeof response.id === "string" ? response.id : `msg_${Date.now()}`;
const model = typeof response.model === "string" ? response.model : "";
const usage = (response.usage as Record<string, unknown>) || {};
const stopReason = typeof response.stop_reason === "string" ? response.stop_reason : "end_turn";
const stopSequence = (response.stop_sequence as string | null | undefined) ?? null;
const content = Array.isArray(response.content) ? response.content : [];
const events: Array<{ event: string; data: Record<string, unknown> }> = [];
events.push({
event: "message_start",
data: {
type: "message_start",
message: {
id: messageId,
type: "message",
role: "assistant",
content: [],
model,
stop_reason: null,
stop_sequence: null,
usage: { input_tokens: usage.input_tokens || 0, output_tokens: 0 },
},
},
});
content.forEach((block: Record<string, unknown>, index: number) => {
if (block.type === "text") {
events.push({
event: "content_block_start",
data: { type: "content_block_start", index, content_block: { type: "text", text: "" } },
});
if (block.text) {
events.push({
event: "content_block_delta",
data: {
type: "content_block_delta",
index,
delta: { type: "text_delta", text: block.text },
},
});
}
events.push({ event: "content_block_stop", data: { type: "content_block_stop", index } });
} else if (block.type === "tool_use") {
events.push({
event: "content_block_start",
data: {
type: "content_block_start",
index,
content_block: { type: "tool_use", id: block.id, name: block.name, input: {} },
},
});
events.push({
event: "content_block_delta",
data: {
type: "content_block_delta",
index,
delta: { type: "input_json_delta", partial_json: JSON.stringify(block.input ?? {}) },
},
});
events.push({ event: "content_block_stop", data: { type: "content_block_stop", index } });
} else if (block.type === "thinking") {
events.push({
event: "content_block_start",
data: { type: "content_block_start", index, content_block: { type: "thinking", thinking: "" } },
});
if (block.thinking) {
events.push({
event: "content_block_delta",
data: {
type: "content_block_delta",
index,
delta: { type: "thinking_delta", thinking: block.thinking },
},
});
}
events.push({ event: "content_block_stop", data: { type: "content_block_stop", index } });
}
});
events.push({
event: "message_delta",
data: {
type: "message_delta",
delta: { stop_reason: stopReason, stop_sequence: stopSequence },
usage: { output_tokens: usage.output_tokens || 0 },
},
});
events.push({ event: "message_stop", data: { type: "message_stop" } });
return events.map((e) => `event: ${e.event}\ndata: ${JSON.stringify(e.data)}\n\n`).join("");
}
export class VertexExecutor extends BaseExecutor {
constructor() {
super("vertex", PROVIDERS.vertex);
}
async execute(input: ExecuteInput) {
const { credentials, log } = input;
const { credentials, log, model, stream } = input;
// Defensive: trim stray surrounding whitespace from a pasted credential.
if (typeof credentials.apiKey === "string") {
credentials.apiKey = credentials.apiKey.trim();
@@ -160,7 +296,53 @@ export class VertexExecutor extends BaseExecutor {
throw err;
}
}
return super.execute(input);
if (isClaudeModel(model) && input.body && typeof input.body === "object") {
let body = input.body as Record<string, unknown>;
if (!Array.isArray(body.messages)) {
body = toAnthropicBody(body);
input.body = body;
}
// The rawPredict endpoint requires "anthropic_version" in the body (Vertex's substitute
// for the "anthropic-version" header used by Anthropic's direct API).
body.anthropic_version ??= "vertex-2023-10-16";
// Unlike Anthropic's direct API (which reads the model from the body), Vertex's
// rawPredict endpoint already encodes project/region/model in the URL and 400s with
// "model: Extra inputs are not permitted" if the translated request body still carries
// one (the openai→claude request translator copies the client's model field over).
delete body.model;
}
const result = await super.execute(input);
if (isClaudeModel(model) && stream) {
const response = result instanceof Response ? result : result?.response;
if (response?.ok) {
const contentType = response.headers.get("content-type") || "";
if (contentType.includes("application/json") && !contentType.includes("text/event-stream")) {
const jsonText = await response.text();
let newBody = jsonText;
let newContentType = contentType;
try {
newBody = synthesizeClaudeSse(JSON.parse(jsonText));
newContentType = "text/event-stream";
} catch (err) {
const message = err instanceof Error ? err.message : String(err);
log?.warn?.("VERTEX", `Failed to synthesize Claude SSE stream: ${message}`);
}
const newHeaders = new Headers(response.headers);
newHeaders.set("content-type", newContentType);
newHeaders.delete("content-length");
const newResponse = new Response(newBody, {
status: response.status,
statusText: response.statusText,
headers: newHeaders,
});
return result instanceof Response ? newResponse : { ...result, response: newResponse };
}
}
}
return result;
}
buildUrl(model: string, stream: boolean, urlIndex = 0, credentials: any = null) {
@@ -189,6 +371,13 @@ export class VertexExecutor extends BaseExecutor {
}
}
if (isClaudeModel(model)) {
// streamRawPredict?alt=sse was verified to return a single plain JSON body (not real SSE
// framing) rather than actual chunked events, which breaks the SSE parser upstream
// ("stream ended before producing a non-ping SSE event"). rawPredict is confirmed reliable
// for both streaming and non-streaming requests; always use it here.
return `https://aiplatform.googleapis.com/v1/projects/${project}/locations/${region}/publishers/anthropic/models/${model}:rawPredict`;
}
if (isPartnerModel(model)) {
return `https://aiplatform.googleapis.com/v1/projects/${project}/locations/global/endpoints/openapi/chat/completions`;
}

View File

@@ -731,7 +731,10 @@ export async function handleChatCore({
// wins; native Claude passthrough is left untouched (it carries its own `thinking`),
// and non-thinking base models are cleaned up later by normalizeThinkingForModel().
// Extracted to chatCore/claudeEffortVariant.ts (#3501); mutates body in place and returns the
// stripped model + an optional log line, keeping behaviour byte-identical.
// stripped model + an optional log line. The strip is unconditional (byte-identical to the
// original behavior) for the claude/Claude-Code-compatible lane; for any other provider it
// additionally requires isKnownClaudeEffortBaseModel(baseModel) to verify the base id is a
// real, effort-capable Claude model before stripping (vertex-claude-catalog-dispatch fix).
{
const effortVariant = applyClaudeEffortVariant({
provider,

View File

@@ -15,6 +15,7 @@
import { splitClaudeEffortSuffix } from "../../config/providerModels.ts";
import { isClaudeCodeCompatibleProvider } from "../../services/claudeCodeCompatible.ts";
import { FORMATS } from "../../translator/formats.ts";
import { isKnownClaudeEffortBaseModel } from "../../utils/claudeEffortVariants.ts";
/**
* True when the client already supplied an explicit reasoning effort (top-level reasoning_effort,
@@ -40,12 +41,10 @@ export function applyClaudeEffortVariant(opts: {
let effectiveModel = opts.effectiveModel;
let log: string | null = null;
if (
(provider === "claude" || isClaudeCodeCompatibleProvider(provider)) &&
typeof effectiveModel === "string"
) {
if (typeof effectiveModel === "string") {
const { baseModel, effort } = splitClaudeEffortSuffix(effectiveModel);
if (effort) {
const isDirectClaudeLane = provider === "claude" || isClaudeCodeCompatibleProvider(provider);
if (effort && (isDirectClaudeLane || isKnownClaudeEffortBaseModel(baseModel))) {
effectiveModel = baseModel;
if (body && typeof body === "object" && !Array.isArray(body)) {
const claudeBody = body as Record<string, unknown>;

View File

@@ -33,7 +33,7 @@ export const CC_DISCOVERY_COMBO_PREFIX = "claude/combo/";
// Ids that already live under the claude/anthropic namespace — never re-mirror them.
const ALREADY_CLAUDE_RE = /^(?:claude|anthropic)(?:\/|$)/i;
// Ids that already carry a reasoning-effort suffix — v1 only mirrors base ids.
const EFFORT_SUFFIX_RE = /-(?:xhigh|high|medium|low)$/i;
const CLAUDE_EFFORT_SUFFIX_RE = /-(?:xhigh|high|medium|low)$/i;
const NO_THINKING_PREFIX = "no-think/";
// Built-in `auto`/`auto/*` combos are synthesized by createBuiltinAutoCombo, NOT
// stored in the DB combos table — the request-path resolver (getComboByName) can't
@@ -66,7 +66,14 @@ function isMirrorableId(id: string): boolean {
if (id.length === 0) return false;
if (ALREADY_CLAUDE_RE.test(id)) return false;
if (id.startsWith(NO_THINKING_PREFIX)) return false;
return !EFFORT_SUFFIX_RE.test(id);
return !CLAUDE_EFFORT_SUFFIX_RE.test(id);
}
/** Strip a `<provider>/` prefix to get the bare model name, matching the convention in
* claudeEffortVariants.ts / noThinkingAlias.ts. */
function bareModelName(id: string): string {
const slash = id.lastIndexOf("/");
return slash >= 0 ? id.slice(slash + 1) : id;
}
export function appendCcDiscoveryAliases<T extends CcDiscoveryCatalogEntry>(
@@ -90,7 +97,10 @@ export function appendCcDiscoveryAliases<T extends CcDiscoveryCatalogEntry>(
aliases.push({
...model,
id: aliasId,
root: id,
// Combo names may legally contain "/" (comboNameSchema allows it), so a combo's
// root must stay the full name verbatim — only real provider-qualified ids get
// the "/" stripped down to the bare model name.
root: isCombo ? id : bareModelName(id),
display_name: `${label} (OmniRoute)`,
} as T);
}

View File

@@ -64,6 +64,17 @@ export function formatClaudeEffortLabel(level: string): string {
return level.charAt(0).toUpperCase() + level.slice(1);
}
/**
* Whether `bareModelId` (no provider prefix, no effort suffix) is a real,
* effort-capable Claude-family model — the single source of truth used both to
* decide whether the catalog should advertise an effort variant AND whether
* dispatch-time stripping should unwind one back to this model.
*/
export function isKnownClaudeEffortBaseModel(bareModelId: string): boolean {
const spec = getModelSpec(bareModelId);
return spec?.supportsThinking === true && CLAUDE_NAME_RE.test(bareModelId);
}
/**
* Whether the catalog should advertise reasoning-effort variants for this entry.
*
@@ -84,10 +95,7 @@ export function shouldExposeClaudeEffortVariants(
if (CLAUDE_EFFORT_SUFFIX_RE.test(id)) return false;
const name = bareModelName(id);
const spec = getModelSpec(name);
if (!spec) return false;
return spec.supportsThinking === true && CLAUDE_NAME_RE.test(name);
return isKnownClaudeEffortBaseModel(name);
}
/**

View File

@@ -31,6 +31,15 @@ import { getModelSpec } from "@/shared/constants/modelSpecs";
export const NO_THINKING_PREFIX = "no-think/";
// Ids that already carry a Claude reasoning-effort suffix (see
// claudeEffortVariants.ts's identical constant) — a no-think variant of an effort
// variant would combine two independent OmniRoute catalog conventions on the same
// id. Dispatch-time, applyNoThinkingAlias pre-sets reasoning_effort:"none" before
// applyClaudeEffortVariant's hasExplicitClaudeEffort() check runs, so the pre-set
// "none" is treated as explicit and the suffix's implied effort is silently
// discarded — semantically incoherent, so never advertise the combination.
const CLAUDE_EFFORT_SUFFIX_RE = /-(?:xhigh|high|medium|low)$/i;
/** True when `modelId` carries the no-thinking gateway prefix. */
export function isNoThinkingAlias(modelId: unknown): modelId is string {
return typeof modelId === "string" && modelId.startsWith(NO_THINKING_PREFIX);
@@ -108,6 +117,7 @@ export function shouldExposeNoThinkingAlias(model: CatalogModelEntry): boolean {
if (typeof id !== "string" || id.length === 0) return false;
if (model.owned_by === "combo") return false; // combos are virtual
if (isNoThinkingAlias(id)) return false; // never double-alias
if (CLAUDE_EFFORT_SUFFIX_RE.test(id)) return false; // never combine with an effort-suffix id
const name = bareModelName(id);
const spec = getModelSpec(name);
@@ -158,7 +168,8 @@ export function appendNoThinkingVariants<T extends CatalogModelEntry>(
const rawId = model.id as string;
const qualifiedId = aliasToCanonical ? normalizeProviderPrefix(rawId, aliasToCanonical) : rawId;
const aliasId = toNoThinkingAlias(qualifiedId);
const variant: T = { ...model, id: aliasId, root: aliasId };
const bareRoot = toNoThinkingAlias(bareModelName(qualifiedId));
const variant: T = { ...model, id: aliasId, root: bareRoot };
if (typeof model.name === "string" && model.name) {
variant.name = `${model.name} (no thinking)`;
}

View File

@@ -35,6 +35,10 @@ function resolvePlaygroundKeyId(
return keys.find((k) => k.key === selectedMaskedKey)?.id ?? null;
}
// Mirrors NO_THINKING_PREFIX in open-sse/utils/noThinkingAlias.ts — kept as a local literal
// (not imported) to avoid pulling server-side catalog modules into the client bundle.
const NO_THINKING_PREFIX = "no-think/";
/**
* Qualify a provider-scoped playground model with its routing prefix so
* OmniRoute can resolve it unambiguously. The previous heuristic only prefixed
@@ -52,6 +56,14 @@ export function qualifyPlaygroundModel(
): string {
const m = (model ?? "").trim();
if (!m || !routingPrefix) return m;
// A no-think id's real wire form is `no-think/<provider>/<model>` — the provider segment
// sits AFTER the prefix, not at the front, so it needs its own qualification branch instead
// of the generic leading-prefix check below.
if (m.startsWith(NO_THINKING_PREFIX)) {
const inner = m.slice(NO_THINKING_PREFIX.length);
const alreadyQualified = inner === routingPrefix || inner.startsWith(`${routingPrefix}/`);
return alreadyQualified ? m : `${NO_THINKING_PREFIX}${routingPrefix}/${inner}`;
}
return m === routingPrefix || m.startsWith(`${routingPrefix}/`) ? m : `${routingPrefix}/${m}`;
}

View File

@@ -1445,6 +1445,7 @@ async function handleSingleModelChat(
comboExecutionKey: runtimeOptions.comboExecutionKey ?? runtimeOptions.comboStepId ?? null,
extendedContext,
modelApiFormat: apiFormat,
modelTargetFormat: targetFormat,
providerProfile,
cachedSettings: runtimeOptions.cachedSettings,
skipUpstreamRetry: runtimeOptions.skipUpstreamRetry ?? false,

View File

@@ -4,14 +4,8 @@ import { connectionHasExtraKeys } from "@omniroute/open-sse/services/apiKeyRotat
import { createBuiltinAutoCombo } from "@omniroute/open-sse/services/autoCombo/builtinCatalog.ts";
import * as log from "../utils/logger";
import { updateProviderCredentials } from "../services/tokenRefresh";
import {
detectFormatFromEndpoint,
getTargetFormat,
} from "@omniroute/open-sse/services/provider.ts";
import {
getModelTargetFormat,
PROVIDER_ID_TO_ALIAS,
} from "@omniroute/open-sse/config/providerModels.ts";
import { detectFormatFromEndpoint } from "@omniroute/open-sse/services/provider.ts";
import { resolveChatCoreTargetFormat } from "@omniroute/open-sse/handlers/chatCore/targetFormat.ts";
import { handleChatCore } from "@omniroute/open-sse/handlers/chatCore.ts";
import {
errorResponse,
@@ -297,12 +291,25 @@ export async function resolveModelOrError(
? ((modelInfo as { apiFormat?: string }).apiFormat as string)
: undefined
: undefined;
const providerAlias = PROVIDER_ID_TO_ALIAS[provider] || provider;
let targetFormat = getModelTargetFormat(providerAlias, model) || getTargetFormat(provider);
if (apiFormat === "responses") {
targetFormat = "openai-responses";
log.info("ROUTING", `Custom model apiFormat=responses → targetFormat=openai-responses`);
}
// customModelTargetFormat: #2905 per-model wire-format override for custom models,
// injected by getModelInfo. Must be threaded into the same resolution formula
// chatCore.ts uses (static registry > custom-model DB override > provider default) —
// a model that's ALSO a static registry entry (e.g. a Vertex Claude model with no
// per-model registry targetFormat) otherwise silently drops the DB override and
// falls through to the provider default, breaking response translation.
const customModelTargetFormat: string | undefined =
modelInfo && typeof modelInfo === "object" && "targetFormat" in modelInfo
? typeof (modelInfo as { targetFormat?: unknown }).targetFormat === "string"
? ((modelInfo as { targetFormat?: string }).targetFormat as string)
: undefined
: undefined;
const { alias: providerAlias, targetFormat } = resolveChatCoreTargetFormat({
provider,
resolvedModel: model,
apiFormat,
customModelTargetFormat,
providerSpecificData: undefined,
});
const ctxTag = extendedContext && providerAlias === "claude" ? " [1m]" : "";
if (modelStr !== `${provider}/${model}`) {
@@ -389,6 +396,7 @@ export async function executeChatWithBreaker({
comboExecutionKey,
extendedContext,
modelApiFormat,
modelTargetFormat,
providerProfile,
cachedSettings,
skipUpstreamRetry = false,
@@ -416,7 +424,19 @@ export async function executeChatWithBreaker({
runWithProxyContext(proxyInfo?.proxy || null, () =>
(handleChatCore as any)({
body: { ...body, model: `${provider}/${model}` },
modelInfo: { provider, model, extendedContext, apiFormat: modelApiFormat },
// #2905-followup: forward the already-resolved custom-model targetFormat
// override through as modelInfo.targetFormat. Without this, chatCore.ts's
// own resolveChatCoreRequestSetup() reads customModelTargetFormat off THIS
// modelInfo object (not the one resolveModelOrError computed it from) and
// finds nothing, silently re-deriving targetFormat from the static registry
// / provider default and discarding the DB override a second time.
modelInfo: {
provider,
model,
extendedContext,
apiFormat: modelApiFormat,
targetFormat: modelTargetFormat,
},
credentials: refreshedCredentials,
log: handlerLog,
clientRawRequest,

View File

@@ -77,6 +77,7 @@ import { isNoAuthProviderBlockedBySettings } from "./noAuthProviderSettings";
import { resolveAccountProxiesFromRegistry } from "./noAuthProxyResolution";
import { getNoAuthHydrationProviderIds } from "./noAuthProviderSiblings";
import { getResource404Bypass } from "./requestResourceHealth";
import { isVertexConnectionWidePermissionDenied } from "./vertexErrorClassifier";
import * as log from "../utils/logger";
import { fisherYatesShuffle, getNextFromDeckSync } from "@/shared/utils/shuffleDeck";
@@ -2146,7 +2147,14 @@ export async function markAccountUnavailable(
: rawCooldownMs;
// ── #3027: per-model subscription/permission 403 → model-only lockout ──
if (isPerModelQuotaProvider && status === 403 && provider && model && !terminalStatus) {
if (
isPerModelQuotaProvider &&
status === 403 &&
provider &&
model &&
!terminalStatus &&
!(provider === "vertex" && isVertexConnectionWidePermissionDenied(errorText))
) {
const lockout = recordModelLockoutFailure(
provider,
connectionId,

View File

@@ -0,0 +1,66 @@
/**
* Google's google.rpc.ErrorInfo proto reliably distinguishes a connection-wide
* PERMISSION_DENIED (API not enabled, or a project-level IAM denial) from a
* model-specific one (IAM denial scoped to a .../models/<id> resource) — see
* https://cloud.google.com/apis/design/errors#error_info. Only returns true on
* POSITIVE evidence of a connection-wide cause; any other shape (including a
* missing/malformed resource field) falls through to the existing per-model
* lockout behavior, since that's the safer default and the actual bug this
* plan fixes (avoid defaulting BACK toward the connection-wide cooldown this
* plan exists to avoid).
*
* Parses the body as JSON and inspects each ErrorInfo-shaped detail object so
* `reason` and `resource` are correlated within the SAME detail entry — a
* multi-detail error body (unusual but possible) must not let one detail's
* resource leak into another detail's reason check. Falls back to a permissive
* regex scan (pre-JSON-parsing behavior) only when the body isn't parseable
* JSON or doesn't contain a `details` array, since Vertex error bodies aren't
* guaranteed to always be well-formed JSON.
*
* Extracted to its own module so the single call site in
* `./auth.ts::markAccountUnavailable()` stays thin wiring, without growing the
* frozen `auth.ts` file (`config/quality/file-size-baseline.json`).
*/
export function isVertexConnectionWidePermissionDenied(
errorText: string | null | undefined
): boolean {
if (!errorText) return false;
try {
const parsed = JSON.parse(errorText);
const details: unknown[] =
parsed?.error?.details ?? parsed?.details ?? (Array.isArray(parsed) ? parsed : []);
if (Array.isArray(details) && details.length > 0) {
for (const detail of details) {
if (!detail || typeof detail !== "object") continue;
const reason = (detail as Record<string, unknown>).reason;
if (reason === "SERVICE_DISABLED") return true;
if (reason === "IAM_PERMISSION_DENIED") {
const metadata = (detail as Record<string, unknown>).metadata;
const resource =
metadata && typeof metadata === "object"
? (metadata as Record<string, unknown>).resource
: undefined;
if (typeof resource === "string" && !resource.includes("/models/")) return true;
}
}
// Well-formed details array present but no detail matched a connection-wide
// pattern (e.g. IAM_PERMISSION_DENIED with a /models/ resource, or no
// recognized reason at all) — per-model lockout is correct, don't fall
// through to the regex heuristic (it would just re-derive the same answer
// less precisely, or worse, could false-positive on stray substrings).
return false;
}
} catch {
// Not parseable JSON — fall through to the regex heuristic below.
}
// Fallback for non-JSON or unexpected-shape error bodies (regex-based,
// pre-JSON-parsing heuristic — kept for robustness against malformed bodies).
if (/"reason"\s*:\s*"SERVICE_DISABLED"/.test(errorText)) return true;
if (/"reason"\s*:\s*"IAM_PERMISSION_DENIED"/.test(errorText)) {
const resourceMatch = errorText.match(/"resource"\s*:\s*"([^"]*)"/);
if (resourceMatch && !resourceMatch[1].includes("/models/")) return true;
}
return false;
}

View File

@@ -39,9 +39,7 @@
"incremental": true,
"incrementalFile": "reports/mutation/stryker-incremental.json",
"testRunner": "tap",
"plugins": [
"@stryker-mutator/tap-runner"
],
"plugins": ["@stryker-mutator/tap-runner"],
"tap": {
"testFiles": [
"tests/unit/7993-noauth-proxy-routing.test.ts",
@@ -307,7 +305,8 @@
"tests/unit/upstream-retry-hints-toggle.test.ts",
"tests/unit/upstream-timeout-model-override.test.ts",
"tests/unit/usage-service-hardening.test.ts",
"tests/unit/validate-response-quality.test.ts"
"tests/unit/validate-response-quality.test.ts",
"tests/unit/vertex-passthrough-model-lockout.test.ts"
],
"nodeArgs": [
"--import",
@@ -416,11 +415,7 @@
".worktrees",
".stryker-tmp"
],
"reporters": [
"progress",
"html",
"json"
],
"reporters": ["progress", "html", "json"],
"htmlReporter": {
"fileName": "reports/mutation/mutation.html"
},

View File

@@ -30,11 +30,25 @@ test("adds a claude/ mirror with display_name and root for an eligible model", (
assert.deepEqual(out[0], models[0]);
const alias = out[1];
assert.equal(alias.id, "claude/kimi/kimi-k2.6");
assert.equal(alias.root, "kimi/kimi-k2.6");
assert.equal(alias.root, "kimi-k2.6");
assert.equal(alias.display_name, "Kimi K2.6 (OmniRoute)");
assert.equal(alias.owned_by, "kimi");
});
test("keeps root bare even when the original id carries a provider prefix", () => {
const models: CatalogEntry[] = [
{ id: "vertex/claude-sonnet-5", owned_by: "vertex", name: "Claude Sonnet 5 (Vertex)" },
];
const out = appendCcDiscoveryAliases(models, alwaysEnabled);
const alias = out.find((m) => m.id === "claude/vertex/claude-sonnet-5");
assert.ok(alias, "mirror entry with the fully-qualified id must exist");
assert.equal(
alias!.root,
"claude-sonnet-5",
"root must be bare, matching the no-think/effort-variant convention"
);
});
test("falls back to the id for display_name when name is missing", () => {
const models: CatalogEntry[] = [{ id: "kimi/kimi-k2.6" }];
const out = appendCcDiscoveryAliases(models, alwaysEnabled);
@@ -86,6 +100,17 @@ test("mirrors combo names containing spaces (comboNameSchema allows them)", () =
assert.equal(out[1].root, "Custo Otimizado BR");
});
test("keeps a combo's root the full name verbatim when the combo name contains a slash", () => {
// comboNameSchema (src/shared/validation/schemas/combo.ts) explicitly allows "/" in
// combo names, so bareModelName must NOT be applied to combo entries — only to real
// provider-qualified model ids.
const models: CatalogEntry[] = [{ id: "Team/Alpha", owned_by: "combo", name: "Team/Alpha" }];
const out = appendCcDiscoveryAliases(models, alwaysEnabled);
assert.equal(out.length, 2);
assert.equal(out[1].id, "claude/combo/Team/Alpha");
assert.equal(out[1].root, "Team/Alpha", "root must be the full combo name, not truncated");
});
test("skips disabled entries and returns the same array reference when nothing is eligible", () => {
const models: CatalogEntry[] = [{ id: "kimi/kimi-k2.6", owned_by: "kimi" }];
const out = appendCcDiscoveryAliases(models, () => false);

View File

@@ -201,6 +201,37 @@ test("resolveModelOrError keeps bare gpt-5.5 on OpenAI when OpenAI is the only a
assert.equal(result.model, "gpt-5.5");
});
test("resolveModelOrError honors a custom-model targetFormat override even when the model id also exists in the static provider registry", async () => {
// #8852-followup: "claude-sonnet-4-6" is a real static registry entry under
// "vertex" (see open-sse/config/providers/registry/vertex/index.ts) with no
// per-model targetFormat, so the provider default ("gemini") normally applies.
// A user who manually added the same id as a custom model with an explicit
// "claude" targetFormat override must have that override win — otherwise
// Vertex's native Anthropic response shape gets mistranslated as Gemini's,
// silently dropping all response content.
await seedConnection("vertex");
const modelsDb = await import("../../src/lib/db/models.ts");
await modelsDb.addCustomModel(
"vertex",
"claude-sonnet-4-6",
"Claude Sonnet 4.6 (Vertex)",
"manual",
"chat-completions",
["chat"],
"claude"
);
const result = await resolveModelOrError(
"vertex/claude-sonnet-4-6",
{ model: "vertex/claude-sonnet-4-6", messages: [{ role: "user", content: "hi" }] },
"/v1/chat/completions"
);
assert.equal(result.provider, "vertex");
assert.equal(result.model, "claude-sonnet-4-6");
assert.equal(result.targetFormat, "claude");
});
test("checkPipelineGates blocks providers with an open circuit breaker", async () => {
const breaker = getCircuitBreaker("openai");
breaker.state = STATE.OPEN;

View File

@@ -2,8 +2,9 @@
// Characterization of applyClaudeEffortVariant — the Claude effort-suffix normalization extracted
// from handleChatCore (chatCore god-file decomposition, #3501). The VS Code "Effort" slider
// advertises claude-...-{low,medium,high,xhigh,max}; Anthropic has no such model, so the suffix is
// stripped to the base id and surfaced as reasoning_effort. Locks: the provider gate (claude /
// claude-code-compatible only), the in-place body mutation (model + reasoning_effort), the
// stripped to the base id and surfaced as reasoning_effort. Locks: the direct-Claude-lane
// unconditional strip (claude / claude-code-compatible), the predicate-gated strip for any other
// provider serving a real Claude model, the in-place body mutation (model + reasoning_effort), the
// sourceFormat==="claude" skip, the explicit-effort-wins rule, and the returned effectiveModel/log.
import { test } from "node:test";
import assert from "node:assert/strict";
@@ -51,7 +52,11 @@ test("sourceFormat 'claude' strips the model but does NOT inject reasoning_effor
});
test("an explicit client reasoning_effort wins (not overwritten)", () => {
const body: Record<string, unknown> = { model: "claude-sonnet-4-low", reasoning_effort: "high", messages: [] };
const body: Record<string, unknown> = {
model: "claude-sonnet-4-low",
reasoning_effort: "high",
messages: [],
};
const r = applyClaudeEffortVariant({
provider: "claude",
effectiveModel: "claude-sonnet-4-low",
@@ -105,3 +110,66 @@ test("non-claude provider is a no-op even with an effort suffix", () => {
assert.equal(body.reasoning_effort, undefined);
assert.equal(r.log, null);
});
test("non-claude provider serving a real Claude model strips the effort suffix", () => {
const body: Record<string, unknown> = { model: "claude-sonnet-5-high", messages: [] };
const r = applyClaudeEffortVariant({
provider: "vertex",
effectiveModel: "claude-sonnet-5-high",
body,
sourceFormat: FORMATS.OPENAI,
});
assert.equal(r.effectiveModel, "claude-sonnet-5");
assert.equal(body.model, "claude-sonnet-5");
assert.equal(body.reasoning_effort, "high");
});
test("safety guard: non-claude provider with a non-Claude model ending in a suffix word is left unchanged", () => {
const body: Record<string, unknown> = { model: "custom-model-high", messages: [] };
const r = applyClaudeEffortVariant({
provider: "some-other-provider",
effectiveModel: "custom-model-high",
body,
sourceFormat: FORMATS.OPENAI,
});
assert.equal(r.effectiveModel, "custom-model-high");
assert.equal(body.model, "custom-model-high");
assert.equal(body.reasoning_effort, undefined);
assert.equal(r.log, null);
});
test("claude-code-compatible provider strips even an unregistered model id (direct lane short-circuits the predicate)", () => {
// Proves the "unconditional strip, zero regression" claim: isDirectClaudeLane short-circuits
// the `||`, so isKnownClaudeEffortBaseModel() is never consulted for claude/CC-compatible
// providers — unlike the safety-guard case above, which requires the predicate to pass.
const body: Record<string, unknown> = {
model: "totally-unregistered-model-xyz-high",
messages: [],
};
const r = applyClaudeEffortVariant({
provider: "anthropic-compatible-cc-default",
effectiveModel: "totally-unregistered-model-xyz-high",
body,
sourceFormat: FORMATS.OPENAI,
});
assert.equal(r.effectiveModel, "totally-unregistered-model-xyz");
assert.equal(body.model, "totally-unregistered-model-xyz");
assert.equal(body.reasoning_effort, "high");
});
test("no-think alias's explicit reasoning_effort:none is not overwritten by a stripped effort suffix", () => {
const body: Record<string, unknown> = {
model: "claude-sonnet-5-high",
reasoning_effort: "none",
messages: [],
};
const r = applyClaudeEffortVariant({
provider: "vertex",
effectiveModel: "claude-sonnet-5-high",
body,
sourceFormat: FORMATS.OPENAI,
});
assert.equal(r.effectiveModel, "claude-sonnet-5");
assert.equal(body.model, "claude-sonnet-5");
assert.equal(body.reasoning_effort, "none");
});

View File

@@ -6,9 +6,12 @@ import {
CLAUDE_XHIGH_EFFORT_LEVEL,
formatClaudeEffortLabel,
shouldExposeClaudeEffortVariants,
isKnownClaudeEffortBaseModel,
claudeEffortLevelsFor,
appendClaudeEffortVariants,
} from "../../open-sse/utils/claudeEffortVariants.ts";
import { shouldExposeNoThinkingAlias } from "../../open-sse/utils/noThinkingAlias.ts";
import { appendCcDiscoveryAliases } from "../../open-sse/utils/ccDiscoveryAliases.ts";
const mk = (id: string, extra: Record<string, unknown> = {}) => ({
id,
@@ -63,6 +66,26 @@ test("non-string / empty / non-object ids never match", () => {
assert.equal(shouldExposeClaudeEffortVariants({ id: 42 as never }), false);
});
// ── isKnownClaudeEffortBaseModel ─────────────────────────────────────────────
test("isKnownClaudeEffortBaseModel returns true for a real effort-capable Claude model", () => {
assert.equal(isKnownClaudeEffortBaseModel("claude-fable-5"), true);
});
test("isKnownClaudeEffortBaseModel returns false for a non-Claude model", () => {
assert.equal(isKnownClaudeEffortBaseModel("gpt-4o"), false);
});
test("isKnownClaudeEffortBaseModel returns false for an unregistered model id", () => {
assert.equal(isKnownClaudeEffortBaseModel("totally-unregistered-model-xyz"), false);
});
test("isKnownClaudeEffortBaseModel returns false for a non-Claude model that also supports thinking (SC-1)", () => {
// gpt-5.5 has supportsThinking:true in MODEL_SPECS (like 36+ other non-Claude models) —
// the /claude/i name check is the only thing excluding it, not the thinking flag alone.
assert.equal(isKnownClaudeEffortBaseModel("gpt-5.5"), false);
});
// ── claudeEffortLevelsFor ────────────────────────────────────────────────────
test("xHigh is added only for models that support it", () => {
@@ -133,3 +156,91 @@ test("never generates variants-of-variants when the list already contains effort
.filter((id) => /-(low|medium|high|xhigh)-(low|medium|high|xhigh)$/.test(id));
assert.deepEqual(doubleSuffixed, []);
});
// ── cross-module drift guard: CLAUDE_EFFORT_SUFFIX_RE parity ────────────────
//
// `CLAUDE_EFFORT_SUFFIX_RE` (`/-(?:xhigh|high|medium|low)$/i`) is intentionally
// duplicated as a local, non-exported constant in THREE sibling modules: this
// file's module (claudeEffortVariants.ts), noThinkingAlias.ts, and
// ccDiscoveryAliases.ts. A cross-import consolidation of that constant was
// already proposed and explicitly reverted earlier in this project's review
// cycle — the plan deliberately kept local duplication for these three
// sibling modules (accepted by the Reduction Analyst). This test does NOT
// argue for reversing that decision and must NOT be read as one. Its only
// purpose is a behavioral drift guard: if a future edit changes the effort
// levels recognized by one copy (e.g. adds a new level, or narrows/widens the
// suffix pattern) without updating the other two, this test fails instead of
// the three modules silently disagreeing about which ids carry an
// effort-level suffix.
test("CLAUDE_EFFORT_SUFFIX_RE stays in sync across claudeEffortVariants/noThinkingAlias/ccDiscoveryAliases (drift guard — do not consolidate, see comment above)", () => {
// Real, registered, thinking-capable Claude model that does NOT reject
// `thinking:{type:"disabled"}` — satisfies every module's registry-lookup
// gate identically, so any behavioral difference below is attributable only
// to the effort-suffix regex, not to some other per-module gating rule.
const BASE = "claude-opus-4-5";
const EFFORT_SUFFIXES = ["-low", "-medium", "-high", "-xhigh", "-XHIGH"];
// Trailing tokens that look suffix-like but must NOT match the regex
// (anchored to exactly low/medium/high/xhigh at end-of-string).
const NON_MATCHING_SUFFIXES = ["-max", "-highest"];
for (const suffix of EFFORT_SUFFIXES) {
const qualifiedId = `claude/${BASE}${suffix}`;
assert.equal(
shouldExposeClaudeEffortVariants(mk(qualifiedId)),
false,
`claudeEffortVariants must exclude ${qualifiedId}`
);
assert.equal(
shouldExposeNoThinkingAlias(mk(qualifiedId)),
false,
`noThinkingAlias must exclude ${qualifiedId}`
);
const mirrored = appendCcDiscoveryAliases(
[{ id: `cc/${BASE}${suffix}`, owned_by: "cc" }],
() => true
);
assert.equal(
mirrored.length,
1,
`ccDiscoveryAliases must never mirror an effort-suffixed id (${suffix})`
);
}
// Control: the identical base model WITHOUT a suffix must pass all three
// gates — proves the suffix itself (not something else about the id) is
// what excluded the cases above.
assert.equal(shouldExposeClaudeEffortVariants(mk(`claude/${BASE}`)), true);
assert.equal(shouldExposeNoThinkingAlias(mk(`claude/${BASE}`)), true);
const baseMirror = appendCcDiscoveryAliases([{ id: `cc/${BASE}`, owned_by: "cc" }], () => true);
assert.equal(baseMirror.length, 2, "unsuffixed id must still be mirrored");
// Suffix-like-but-non-matching trailing tokens must NOT be excluded by the
// regex. This isolates the regex's specificity (exactly xhigh/high/medium/low)
// from the models-registry prefix-matching gate: `getCanonicalModelSpecId`
// resolves "claude-opus-4-5-max" back to the "claude-opus-4-5" spec via its
// prefix-match fallback, so `shouldExposeClaudeEffortVariants` /
// `shouldExposeNoThinkingAlias` still pass their registry-lookup gate here —
// any exclusion left could only come from the suffix regex, and there is none.
for (const suffix of NON_MATCHING_SUFFIXES) {
const qualifiedId = `claude/${BASE}${suffix}`;
assert.equal(
shouldExposeClaudeEffortVariants(mk(qualifiedId)),
true,
`claudeEffortVariants must not treat "${suffix}" as an effort suffix`
);
assert.equal(
shouldExposeNoThinkingAlias(mk(qualifiedId)),
true,
`noThinkingAlias must not treat "${suffix}" as an effort suffix`
);
const mirrored = appendCcDiscoveryAliases(
[{ id: `cc/${BASE}${suffix}`, owned_by: "cc" }],
() => true
);
assert.equal(
mirrored.length,
2,
`ccDiscoveryAliases must still mirror a non-effort-suffix-looking id ("${suffix}")`
);
}
});

View File

@@ -90,11 +90,15 @@ test("VertexExecutor.buildUrl routes partner and org-prefixed models to the glob
);
});
test("VertexExecutor.buildUrl routes current-generation Claude models to the global partner endpoint (#1985)", () => {
test("VertexExecutor.buildUrl routes current-generation Claude models to the native Anthropic rawPredict endpoint (#1985)", () => {
const executor = new VertexExecutor();
// These model IDs post-date the old pinned "claude-3-5-sonnet" / "claude-3-opus" /
// "claude-3-haiku" prefixes and were previously misrouted to the Google-publisher path.
// "claude-3-haiku" prefixes and were previously misrouted to the Google-publisher path,
// then (once generalized to a "claude-" prefix, #1985) to the generic OpenAI-compatible
// partner endpoint. Claude models use Vertex's native Anthropic Messages API
// (publishers/anthropic/.../rawPredict) instead — the partner endpoint 404s/"malformed
// argument"s for Claude on at least some projects.
const claude4Sonnet = executor.buildUrl("claude-sonnet-4-6", false, 0, {
apiKey: createServiceAccountJson({ projectId: "proj-claude" }),
});
@@ -104,11 +108,11 @@ test("VertexExecutor.buildUrl routes current-generation Claude models to the glo
assert.equal(
claude4Sonnet,
"https://aiplatform.googleapis.com/v1/projects/proj-claude/locations/global/endpoints/openapi/chat/completions"
"https://aiplatform.googleapis.com/v1/projects/proj-claude/locations/us-central1/publishers/anthropic/models/claude-sonnet-4-6:rawPredict"
);
assert.equal(
claude4Haiku,
"https://aiplatform.googleapis.com/v1/projects/proj-claude/locations/global/endpoints/openapi/chat/completions"
"https://aiplatform.googleapis.com/v1/projects/proj-claude/locations/us-central1/publishers/anthropic/models/claude-haiku-4-5@20251001:rawPredict"
);
});
@@ -238,3 +242,110 @@ test("VertexExecutor.execute rejects incomplete Service Account JSON clearly", a
/missing required fields/
);
});
test("VertexExecutor.execute strips the client's model field and injects anthropic_version for Claude models", async () => {
const executor = new VertexExecutor();
const originalFetch = globalThis.fetch;
const calls = [];
globalThis.fetch = async (url, options) => {
calls.push({ url: String(url), body: String(options?.body || "") });
return new Response(
JSON.stringify({
id: "msg_1",
type: "message",
role: "assistant",
model: "claude-sonnet-4-6",
content: [{ type: "text", text: "hi" }],
stop_reason: "end_turn",
usage: { input_tokens: 3, output_tokens: 1 },
}),
{ status: 200, headers: { "Content-Type": "application/json" } }
);
};
try {
await executor.execute({
model: "claude-sonnet-4-6",
// rawPredict rejects a body-level "model" field ("Extra inputs are not permitted") since
// the model is already encoded in the URL — the openai→claude request translator copies
// the client's model field over, so the executor must strip it before sending.
body: { model: "vertex/claude-sonnet-4-6", messages: [{ role: "user", content: "hi" }] },
stream: false,
credentials: {
apiKey: createServiceAccountJson({ projectId: "proj-claude" }),
accessToken: "ya29.claude",
},
});
assert.equal(calls.length, 1);
const sentBody = JSON.parse(calls[0].body);
assert.equal(sentBody.model, undefined);
assert.equal(sentBody.anthropic_version, "vertex-2023-10-16");
assert.deepEqual(sentBody.messages, [{ role: "user", content: "hi" }]);
} finally {
globalThis.fetch = originalFetch;
}
});
test("VertexExecutor.execute synthesizes a genuine Anthropic-format SSE stream when rawPredict returns a complete JSON body for a streaming request", async () => {
const executor = new VertexExecutor();
const originalFetch = globalThis.fetch;
// rawPredict is a non-streaming endpoint — Vertex can still hand back a complete,
// non-chunked JSON body for a request that asked for stream:true. Without synthesis
// this reaches the client as a single JSON blob the OpenAI-only jsonToSse fallback
// can't parse (it looks for "choices", not Anthropic's "content" shape), producing
// "Provider returned empty content" instead of real streamed text.
globalThis.fetch = async () =>
new Response(
JSON.stringify({
id: "msg_stream_1",
type: "message",
role: "assistant",
model: "claude-sonnet-4-6",
content: [{ type: "text", text: "hello" }],
stop_reason: "end_turn",
stop_sequence: null,
usage: { input_tokens: 5, output_tokens: 2 },
}),
{ status: 200, headers: { "Content-Type": "application/json" } }
);
try {
const result = await executor.execute({
model: "claude-sonnet-4-6",
body: { messages: [{ role: "user", content: "hi" }] },
stream: true,
credentials: {
apiKey: createServiceAccountJson({ projectId: "proj-claude" }),
accessToken: "ya29.claude",
},
});
const response = result.response;
assert.equal(response.status, 200);
assert.equal(response.headers.get("content-type"), "text/event-stream");
const text = await response.text();
assert.match(text, /event: message_start/);
assert.match(text, /"type":"content_block_delta".*"text":"hello"/);
assert.match(text, /event: message_stop/);
const dataLines = text
.split(/\r?\n/)
.filter((line) => line.startsWith("data:"))
.map((line) => JSON.parse(line.slice(5).trim()));
const types = dataLines.map((d) => d.type);
assert.deepEqual(types, [
"message_start",
"content_block_start",
"content_block_delta",
"content_block_stop",
"message_delta",
"message_stop",
]);
} finally {
globalThis.fetch = originalFetch;
}
});

View File

@@ -73,7 +73,11 @@ test("applyNoThinkingAlias expresses reasoning_effort:none without a thinking bl
// #6879: a thinks-by-default OpenAI-shape model must carry reasoning_effort:"none"
// explicitly (not merely have the field deleted), so suppression actually takes
// effect downstream; the Responses-shaped `reasoning` object is still dropped.
assert.equal(body.reasoning_effort, "none", "reasoning_effort must express none, not be stripped");
assert.equal(
body.reasoning_effort,
"none",
"reasoning_effort must express none, not be stripped"
);
assert.ok(!("reasoning" in body), "reasoning object must be dropped");
});
@@ -96,11 +100,7 @@ test("applyNoThinkingAlias ignores a malformed prefix-only model", () => {
const body: Record<string, unknown> = { model: "no-think/" };
const res = applyNoThinkingAlias(body, { claudeFormat: true });
assert.equal(res.applied, false);
assert.equal(
body.model,
"no-think/",
"left untouched when nothing follows the prefix"
);
assert.equal(body.model, "no-think/", "left untouched when nothing follows the prefix");
});
// ── catalog gating ───────────────────────────────────────────────────────────
@@ -120,28 +120,16 @@ test("shouldExposeNoThinkingAlias rejects models where suppression is meaningles
// combos are virtual, never aliased
assert.equal(shouldExposeNoThinkingAlias(entry("my-combo", "combo")), false);
// never double-alias
assert.equal(
shouldExposeNoThinkingAlias(entry("no-think/anthropic/claude-opus-4-5")),
false
);
assert.equal(shouldExposeNoThinkingAlias(entry("no-think/anthropic/claude-opus-4-5")), false);
});
test("appendNoThinkingVariants adds one variant per eligible model and preserves the rest", () => {
const models = [entry("claude-opus-4-5"), entry("gpt-4o", "openai"), entry("claude-fable-5")];
const out = appendNoThinkingVariants(models);
const ids = out.map((m) => m.id);
assert.ok(
ids.includes("no-think/claude-opus-4-5"),
"eligible model gets a variant"
);
assert.ok(
!ids.includes("no-think/gpt-4o"),
"non-thinking model has no variant"
);
assert.ok(
!ids.includes("no-think/claude-fable-5"),
"reject-disabled model has no variant"
);
assert.ok(ids.includes("no-think/claude-opus-4-5"), "eligible model gets a variant");
assert.ok(!ids.includes("no-think/gpt-4o"), "non-thinking model has no variant");
assert.ok(!ids.includes("no-think/claude-fable-5"), "reject-disabled model has no variant");
assert.equal(out.length, models.length + 1, "exactly one variant appended");
// originals preserved up front
assert.deepEqual(out.slice(0, 3), models);
@@ -157,22 +145,42 @@ test("appendNoThinkingVariants normalizes alias prefix to canonical when aliasTo
const aliasToCanonical = { cc: "claude" };
const out = appendNoThinkingVariants(models, aliasToCanonical);
const ids = out.map((m) => m.id);
assert.ok(
ids.includes("no-think/claude/claude-opus-4-5"),
"uses canonical prefix"
);
assert.ok(
!ids.includes("no-think/cc/claude-opus-4-5"),
"alias prefix not used"
);
assert.ok(ids.includes("no-think/claude/claude-opus-4-5"), "uses canonical prefix");
assert.ok(!ids.includes("no-think/cc/claude-opus-4-5"), "alias prefix not used");
});
test("appendNoThinkingVariants keeps alias prefix when no map is provided", () => {
const models = [entry("cc/claude-opus-4-5")];
const out = appendNoThinkingVariants(models);
const ids = out.map((m) => m.id);
assert.ok(ids.includes("no-think/cc/claude-opus-4-5"), "alias prefix preserved");
});
test("appendNoThinkingVariants keeps root bare even when id carries a provider prefix", () => {
const models = [entry("vertex/claude-opus-4-5", "vertex")];
const out = appendNoThinkingVariants(models);
const variant = out.find((m) => m.id === "no-think/vertex/claude-opus-4-5");
assert.ok(variant, "variant with the fully-qualified id must exist");
assert.equal(
variant!.root,
"no-think/claude-opus-4-5",
"root must be bare (no embedded provider segment), matching the effort-variant convention"
);
});
test("shouldExposeNoThinkingAlias rejects an already effort-suffixed id", () => {
assert.equal(shouldExposeNoThinkingAlias(entry("vertex/claude-sonnet-5-high")), false);
assert.equal(shouldExposeNoThinkingAlias(entry("claude-opus-4-5-xhigh")), false);
});
test("appendNoThinkingVariants does not synthesize a no-think variant of an effort variant", () => {
// Simulates the real pipeline order in catalogResponse.ts: appendClaudeEffortVariants
// runs first and produces an id like this before appendNoThinkingVariants ever sees it.
const models = [entry("vertex/claude-sonnet-5-high")];
const out = appendNoThinkingVariants(models);
assert.equal(out, models, "no variant should be added for an effort-suffixed id");
assert.ok(
ids.includes("no-think/cc/claude-opus-4-5"),
"alias prefix preserved"
!out.some((m) => m.id === "no-think/vertex/claude-sonnet-5-high"),
"the incoherent combined id must never be advertised"
);
});

View File

@@ -37,3 +37,37 @@ test("OpenCode Free playground uses its routing alias instead of the reserved pr
assert.equal(getProviderAlias("opencode"), "oc");
assert.equal(qualifyPlaygroundModel("big-pickle", getProviderAlias("opencode")), "oc/big-pickle");
});
test("qualifyPlaygroundModel inserts the provider after the no-think prefix, not before it", () => {
assert.equal(
qualifyPlaygroundModel("no-think/claude-sonnet-5", "vertex"),
"no-think/vertex/claude-sonnet-5"
);
});
test("qualifyPlaygroundModel does not double-qualify an already-qualified no-think id", () => {
assert.equal(
qualifyPlaygroundModel("no-think/vertex/claude-sonnet-5", "vertex"),
"no-think/vertex/claude-sonnet-5"
);
});
test("qualifyPlaygroundModel does not mistake a provider-name-prefix collision for already-qualified", () => {
// routingPrefix "vertex" must not match "vertex-eu/..." as already-qualified just because
// it starts with the same characters — the check requires an exact "vertex/" segment
// boundary. A naive `inner.startsWith(routingPrefix)` (no slash) would wrongly skip
// qualification here and leave the provider segment un-inserted.
assert.equal(
qualifyPlaygroundModel("no-think/vertex-eu/claude-sonnet-5", "vertex"),
"no-think/vertex/vertex-eu/claude-sonnet-5"
);
});
test("LlmChatCard's local NO_THINKING_PREFIX literal matches the canonical constant", async () => {
// Drift guard: LlmChatCard.tsx deliberately hardcodes "no-think/" as a literal instead
// of importing NO_THINKING_PREFIX from open-sse/utils/noThinkingAlias.ts (avoids pulling
// server-side catalog modules into the client bundle — see Step 1). This test file is not
// client-bundled, so it can safely import the real constant and assert they never drift.
const { NO_THINKING_PREFIX } = await import("../../open-sse/utils/noThinkingAlias.ts");
assert.equal(NO_THINKING_PREFIX, "no-think/");
});

View File

@@ -0,0 +1,292 @@
// Regression guard: after adding passthroughModels: true to Vertex's registry entry, a 404 on
// one Vertex model (e.g. a stale/synthetic model id) must lock out only that model, not cool
// down the whole connection — mirrors the existing ollama-cloud/bedrock protection. Before this
// fix, hasPerModelQuota("vertex", ...) was false, so any 404 on Vertex cooled the whole
// connection for COOLDOWN_MS.notFound (2 minutes), per errorConfig.ts's generic status_404 rule.
import test from "node:test";
import assert from "node:assert/strict";
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-vertex-404-"));
process.env.DATA_DIR = TEST_DATA_DIR;
const core = await import("../../src/lib/db/core.ts");
const providersDb = await import("../../src/lib/db/providers.ts");
const auth = await import("../../src/sse/services/auth.ts");
const accountFallback = await import("../../open-sse/services/accountFallback.ts");
async function resetStorage() {
core.resetDbInstance();
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
fs.mkdirSync(TEST_DATA_DIR, { recursive: true });
}
async function seedVertex() {
return providersDb.createProviderConnection({
provider: "vertex",
authType: "apikey",
apiKey: "vertex-key",
isActive: true,
testStatus: "active",
});
}
test.after(() => {
core.resetDbInstance();
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
});
test('hasPerModelQuota("vertex", ...) is true after the passthroughModels registry flag', () => {
assert.equal(accountFallback.hasPerModelQuota("vertex", "claude-sonnet-5"), true);
});
test("404 on one Vertex model locks only that model, connection stays active", async () => {
await resetStorage();
const conn = await seedVertex();
const result = await auth.markAccountUnavailable(
conn.id,
404,
"model not found",
"vertex",
"claude-sonnet-5-high"
);
assert.equal(result.shouldFallback, true);
const after = await providersDb.getProviderConnectionById(conn.id);
assert.equal(after.testStatus, "active");
assert.ok(!after.rateLimitedUntil, "connection must not be rate-limited");
const lockout = accountFallback.getModelLockoutInfo("vertex", conn.id, "claude-sonnet-5-high");
assert.equal(lockout?.reason, "not_found");
// A sibling model on the same connection must remain immediately eligible.
const sibling = accountFallback.getModelLockoutInfo("vertex", conn.id, "gemini-3.1-pro-preview");
assert.equal(sibling, null);
});
test("503 on one Vertex model locks only that model, connection stays active (proves the fix isn't 404-specific)", async () => {
// hasPerModelQuota's gate covers status === 404 || status === 429 || status >= 500
// (auth.ts:2024) in one shared branch — 502/503/504 keep the model-lockout path (only
// the exact 500 is exempted per #5976). This mirrors the 404 test above with a 5xx to
// confirm passthroughModels doesn't just fix the specific 404 symptom reported.
await resetStorage();
const conn = await seedVertex();
const result = await auth.markAccountUnavailable(
conn.id,
503,
"Service Unavailable",
"vertex",
"claude-sonnet-5-high"
);
assert.equal(result.shouldFallback, true);
const after = await providersDb.getProviderConnectionById(conn.id);
assert.equal(after.testStatus, "active");
assert.ok(!after.rateLimitedUntil, "connection must not be rate-limited");
const lockout = accountFallback.getModelLockoutInfo("vertex", conn.id, "claude-sonnet-5-high");
assert.equal(lockout?.reason, "server_error");
const sibling = accountFallback.getModelLockoutInfo("vertex", conn.id, "gemini-3.1-pro-preview");
assert.equal(sibling, null);
});
test("403 PERMISSION_DENIED on Vertex locks only that model too (accepted trade-off, see plan)", async () => {
await resetStorage();
const conn = await seedVertex();
// Google Cloud uses the literal "PERMISSION_DENIED" status name for BOTH a
// model-specific denial and a connection-wide IAM/API-disabled failure — this
// fix cannot distinguish them (no live Vertex credential test in this plan), so
// it intentionally treats both as a per-model lockout post-passthroughModels.
const result = await auth.markAccountUnavailable(
conn.id,
403,
"PERMISSION_DENIED: the caller does not have permission",
"vertex",
"claude-sonnet-5-high"
);
assert.equal(result.shouldFallback, true);
const after = await providersDb.getProviderConnectionById(conn.id);
assert.equal(after.testStatus, "active");
assert.ok(!after.rateLimitedUntil, "connection must not be rate-limited");
const lockout = accountFallback.getModelLockoutInfo("vertex", conn.id, "claude-sonnet-5-high");
assert.equal(lockout?.reason, "forbidden");
});
// ── 403 disambiguation via google.rpc.ErrorInfo (Task 7) ────────────────────
// Google Cloud's documented error format (https://cloud.google.com/apis/design/errors#error_info)
// lets us tell a genuinely connection-wide PERMISSION_DENIED (API disabled, or a
// project-level IAM denial) apart from one scoped to a single model — the former must
// fall through to the existing connection-wide cooldown instead of the #3027 per-model
// lockout path, since a per-model lockout would leave a broken connection "active".
test("403 with SERVICE_DISABLED ErrorInfo reason cools down the whole Vertex connection", async () => {
await resetStorage();
const conn = await seedVertex();
const body = JSON.stringify({
error: {
status: "PERMISSION_DENIED",
details: [
{
"@type": "type.googleapis.com/google.rpc.ErrorInfo",
reason: "SERVICE_DISABLED",
domain: "googleapis.com",
metadata: { service: "aiplatform.googleapis.com", consumer: "projects/12345" },
},
],
},
});
const result = await auth.markAccountUnavailable(
conn.id,
403,
body,
"vertex",
"claude-sonnet-5-high"
);
assert.equal(result.shouldFallback, true);
const after = await providersDb.getProviderConnectionById(conn.id);
assert.equal(after.testStatus, "unavailable");
assert.ok(after.rateLimitedUntil, "connection must be cooled down, not left active");
// The #3027 per-model lockout path must NOT have run.
const lockout = accountFallback.getModelLockoutInfo("vertex", conn.id, "claude-sonnet-5-high");
assert.equal(lockout, null);
});
test("403 with IAM_PERMISSION_DENIED reason and a model-scoped resource still locks only that model", async () => {
await resetStorage();
const conn = await seedVertex();
const body = JSON.stringify({
error: {
status: "PERMISSION_DENIED",
details: [
{
"@type": "type.googleapis.com/google.rpc.ErrorInfo",
reason: "IAM_PERMISSION_DENIED",
domain: "iam.googleapis.com",
metadata: {
permission: "aiplatform.endpoints.predict",
resource:
"projects/12345/locations/us-central1/publishers/google/models/claude-sonnet-5",
},
},
],
},
});
const result = await auth.markAccountUnavailable(
conn.id,
403,
body,
"vertex",
"claude-sonnet-5-high"
);
assert.equal(result.shouldFallback, true);
const after = await providersDb.getProviderConnectionById(conn.id);
assert.equal(after.testStatus, "active");
assert.ok(!after.rateLimitedUntil, "connection must not be rate-limited");
const lockout = accountFallback.getModelLockoutInfo("vertex", conn.id, "claude-sonnet-5-high");
assert.equal(lockout?.reason, "forbidden");
});
test("403 with IAM_PERMISSION_DENIED reason and a project-level resource cools down the whole connection", async () => {
await resetStorage();
const conn = await seedVertex();
const body = JSON.stringify({
error: {
status: "PERMISSION_DENIED",
details: [
{
"@type": "type.googleapis.com/google.rpc.ErrorInfo",
reason: "IAM_PERMISSION_DENIED",
domain: "iam.googleapis.com",
metadata: {
permission: "aiplatform.googleapis.com/models.predict",
resource: "projects/12345",
},
},
],
},
});
const result = await auth.markAccountUnavailable(
conn.id,
403,
body,
"vertex",
"claude-sonnet-5-high"
);
assert.equal(result.shouldFallback, true);
const after = await providersDb.getProviderConnectionById(conn.id);
assert.equal(after.testStatus, "unavailable");
assert.ok(after.rateLimitedUntil, "connection must be cooled down, not left active");
const lockout = accountFallback.getModelLockoutInfo("vertex", conn.id, "claude-sonnet-5-high");
assert.equal(lockout, null);
});
test("multi-detail error body correlates reason+resource per-detail, not across details", async () => {
// Adversarial case: detail[0] has a model-scoped resource under an unrelated reason,
// detail[1] carries the actual IAM_PERMISSION_DENIED with a project-level resource. A
// naive independent-regex scan would match detail[0]'s resource against detail[1]'s
// reason and wrongly conclude "model-scoped" — this must resolve to connection-wide.
await resetStorage();
const conn = await seedVertex();
const body = JSON.stringify({
error: {
status: "PERMISSION_DENIED",
details: [
{
"@type": "type.googleapis.com/google.rpc.ErrorInfo",
reason: "SOME_OTHER_REASON",
domain: "iam.googleapis.com",
metadata: {
resource:
"projects/12345/locations/us-central1/publishers/google/models/claude-sonnet-5",
},
},
{
"@type": "type.googleapis.com/google.rpc.ErrorInfo",
reason: "IAM_PERMISSION_DENIED",
domain: "iam.googleapis.com",
metadata: {
permission: "aiplatform.googleapis.com/models.predict",
resource: "projects/12345",
},
},
],
},
});
const result = await auth.markAccountUnavailable(
conn.id,
403,
body,
"vertex",
"claude-sonnet-5-high"
);
assert.equal(result.shouldFallback, true);
const after2 = await providersDb.getProviderConnectionById(conn.id);
assert.equal(after2.testStatus, "unavailable");
assert.ok(after2.rateLimitedUntil, "connection must be cooled down, not left active");
const lockout2 = accountFallback.getModelLockoutInfo("vertex", conn.id, "claude-sonnet-5-high");
assert.equal(lockout2, null);
});