mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-05 06:42:12 +03:00
fix(sse): adapt the Claude flow to Opus 4.7+ adaptive-thinking + fixed-sampling contract (#4230)
* feat(sse): adapt the Claude flow to Opus 4.7+ adaptive-thinking + fixed-sampling contract
Claude Opus 4.7 and later (Opus 4.7/4.8, Fable 5) changed the Messages API contract:
manual extended thinking was removed (`thinking.type:"enabled"` or ANY `budget_tokens`
returns 400 — adaptive-only, steered by `output_config.effort`) and non-default
`temperature`/`top_p`/`top_k` now return 400 (sampling is fixed). OmniRoute still emitted
both shapes, so OpenAI-format `reasoning_effort` low/medium/high and any client-supplied
sampling param could hard-400 on the most-used provider.
- modelSpecs: new `adaptiveThinkingOnly` flag + `isAdaptiveThinkingOnly()` on opus-4-7/4-8/fable-5
- translator: `reasoning_effort` (every level) -> adaptive + `output_config.effort` for those models
- chatCore: `normalizeClaudeAdaptiveThinking` catch-all collapses residual manual thinking
(passthrough legacy shape / per-model defaults) to `{type:"adaptive"}`, keyed on the model
- registry: strip `temperature`/`top_p`/`top_k` for opus-4-7/4-8/fable-5 (claude + anthropic ids)
Pre-4.7 models (Opus 4.6/4.5, Sonnet, Haiku) keep manual budgets and sampling params.
TDD: +27 cases (adaptive-thinking-normalize, sampling-params, translator). typecheck/lint/file-size green.
* docs(changelog): note Claude Opus 4.7+ adaptive-thinking + fixed-sampling fixes (#4230)
This commit is contained in:
committed by
GitHub
parent
8b73b37d09
commit
a044246f9b
@@ -21,6 +21,8 @@ _In development — bullets added per PR; finalized at release._
|
||||
- **fix(capabilities): resolve models.dev-synced vision metadata for Mistral `-latest` aliases** — root cause behind the #4071 heuristic: `getResolvedModelCapabilities("mistral/pixtral-12b-latest").supportsVision` resolved `null` (vision came only from the #4071 model-id heuristic, with `attachment` still `null`) even though models.dev exposes the model as multimodal. Confirmed against the live models.dev API: it catalogs Pixtral 12B under the **short** id `pixtral-12b` (with `attachment: true`, `modalities.input: ["text","image"]`), while requests use the Mistral API alias `pixtral-12b-latest`. The synced lookup tried the exact / raw / static-spec-canonical ids — all of which miss the short form — so it fell through to the heuristic. `getSyncedCapabilityForResolved` now adds a last-resort fallback that retries with a trailing `-latest` stripped, so synced metadata (`attachment` / image modalities) wins for these aliases; models whose `-latest` id is stored verbatim (e.g. `pixtral-large-latest`) keep resolving directly. Note: the models.dev sync is currently manual-only (Settings → models.dev) with no scheduled refresh, so a fresh instance still relies on the #4071 heuristic until that sync runs — a periodic-refresh cadence is left as a separate follow-up. ([#4073](https://github.com/diegosouzapw/OmniRoute/issues/4073) — thanks @diego-anselmo)
|
||||
- **fix(sse): map Xiaomi MiMo reasoning control to its native `thinking:{type}` shape** — MiMo (`api.xiaomimimo.com`) controls chain-of-thought **only** via top-level `thinking:{type:"enabled"|"disabled"}` and does not understand OpenAI's `reasoning_effort`/`reasoning`, while its request validator is strict (`400 Param Incorrect`). OmniRoute's OpenAI path carried reasoning intent as `reasoning_effort`, and the claude→openai translator can leave a Claude-shaped `thinking:{type, budget_tokens}` — so the client's on/off choice was silently dropped and `budget_tokens`/`reasoning_effort` rode along as extra params the validator can reject. New `open-sse/services/mimoThinking.ts::normalizeMimoThinking` (wired in `chatCore` for `provider==="xiaomi-mimo"`) reduces any thinking object to just `{type}` (`disabled` stays; `enabled`/`adaptive`/other → `enabled`) and drops `reasoning_effort`/`reasoning`. It deliberately does **not** synthesize thinking from a bare `reasoning_effort` — `mimo-v2-omni` is non-thinking, so that could turn a silently-ignored param into a hard error. ([#4224](https://github.com/diegosouzapw/OmniRoute/pull/4224))
|
||||
- **fix(capabilities): Xiaomi MiMo `*-pro` chat models are text-only (no vision)** — only `mimo-v2.5` and `mimo-v2-omni` accept images per Xiaomi's docs; `mimo-v2.5-pro`/`mimo-v2-pro` are text-only, but `modelSpecs` marked them vision-capable and models.dev mislabels them ([hermes-agent#18884](https://github.com/NousResearch/hermes-agent/issues/18884)). Since `resolveVisionCapability` lets a synced `attachment:true` win first, an image request could be routed to a blind model (the #4071 failure mode). Corrected the specs **and** added a hard override in `resolveVisionCapability` (checked before the synced branch, anchored so `mimo-v2.5-pro` never matches the multimodal `mimo-v2.5`) that beats the wrong synced attachment. Also registered the missing native `mimo-v2-pro` chat model and the missing `mimo-v2-tts` speech model. ([#4224](https://github.com/diegosouzapw/OmniRoute/pull/4224))
|
||||
- **fix(sse): Claude Opus 4.7+/Fable 5 use adaptive thinking only (no more manual-budget 400s)** — Opus 4.7 and later (Opus 4.7/4.8, Fable 5) removed manual extended thinking: `thinking.type:"enabled"` or **any** `thinking.budget_tokens` now returns `400` ("Any request that tries to set a fixed thinking budget gets a 400" — Anthropic migration guide). Reasoning is adaptive-only, steered by `output_config.effort`. OmniRoute's OpenAI→Claude translator mapped `reasoning_effort` low/medium/high to a manual `thinking:{type:"enabled", budget_tokens}`, so those requests hard-400'd on the most-used provider (and a Claude-native passthrough client sending the legacy shape did too). A new `adaptiveThinkingOnly` model flag now drives two fixes: the translator maps `reasoning_effort` of **every** level to `{type:"adaptive"}` + `output_config.effort` (preserving the requested level, never a budget) for these models, and a `normalizeClaudeAdaptiveThinking` catch-all at the existing post-translation thinking-normalization chokepoint collapses any residual manual thinking (passthrough legacy shape, per-model defaults) to `{type:"adaptive"}`, keyed on the resolved upstream model so it covers every routing mode. Pre-4.7 models (Opus 4.6/4.5, Sonnet, Haiku) keep manual budgets unchanged. ([#4230](https://github.com/diegosouzapw/OmniRoute/pull/4230))
|
||||
- **fix(providers): strip non-default temperature/top_p/top_k for Claude Opus 4.7+/Fable 5 (fixed sampling → no 400)** — Opus 4.7 and later reject non-default `temperature`/`top_p`/`top_k` with a `400` (sampling is fixed; reasoning moved to `output_config.effort`). The translator forwarded client-supplied `temperature`/`top_p` unconditionally and the Claude registry models carried no `unsupportedParams`, so a plain OpenAI-format request with `temperature: 0.7` to `claude-opus-4-8` hard-400'd. Added `unsupportedParams: ["temperature","top_p","top_k"]` to the Opus 4.7+/Fable 5 ids in both the `claude` (dashed `claude-opus-4-8`) and `anthropic` (dotted `claude-opus-4.7`) registries, so they're stripped at the existing `getUnsupportedParams` dispatch chokepoint. Pre-4.7 Claude models still accept sampling params. ([#4230](https://github.com/diegosouzapw/OmniRoute/pull/4230))
|
||||
|
||||
### 🧪 Tests
|
||||
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
{
|
||||
"_comment": "Catraca de tamanho (check-file-size.mjs). frozen so pode encolher; arquivos novos <= cap. --update ratcheta.",
|
||||
"_rebaseline_2026_06_18_claude_adaptive_thinking": "PR (Claude Opus 4.7+ adaptive-thinking flow) own growth: chatCore.ts 5095->5102 (+7 = wire normalizeClaudeAdaptiveThinking(translatedBody, finalModelToUpstream) at the existing post-translation thinking-normalization chokepoint, right after normalizeThinkingForModel — its import line + a 5-line explanatory comment + the call). Opus 4.7+/Fable 5 removed manual extended thinking: `thinking.type:\"enabled\"` or any `thinking.budget_tokens` is a hard 400, so this collapses any manual thinking that reached dispatch (passthrough legacy shape, reasoning_effort buckets, per-model defaults) to `{type:\"adaptive\"}`, keyed on the resolved upstream model. The reusable guard lives in the new pure leaf open-sse/services/claudeAdaptiveThinking.ts (<cap), mirroring normalizeMimoThinking (#4224). Cohesive at the existing thinking-normalization chokepoint, next to normalizeThinkingForModel/normalizeMimoThinking; not extractable without hiding the normalization boundary. Structural shrink of chatCore.ts tracked in #3501.",
|
||||
"_rebaseline_2026_06_18_4221_tool_cardinality": "PR #4221 own growth: server.ts 1458->1468 (+10 at the existing registerTool override in createMcpServer = F4.3 opt-in tool-cardinality wiring). Reads readMcpToolProfileFromEnv(process.env) once (MCP_TOOL_DENY/MCP_TOOL_ALLOW; null = no filter, the default), and when a registered tool is denied by the profile calls registered.disable() so it is not announced in tools/list (token savings). The default (null) profile never enters the branch — existing behavior byte-identical. The reusable parser + reduceToolManifest decision live in the (non-frozen) toolCardinality.ts. Cohesive opt-in feature at the registration chokepoint; not extractable without hiding the register boundary.",
|
||||
"_rebaseline_2026_06_18_4217_compression_step_streaming": "PR #4217 own growth: chatCore.ts 5063->5086 (+23 at the existing compression-apply chokepoint = the best-effort onEngineStep callback threaded into applyCompressionAsync). The callback builds a compression.step payload and fires emit(\"compression.step\", …) + forwardDashboardEventToLiveWs(…) once per stacked engine as it completes (F3.3 live per-engine streaming), wrapped in try/catch so it never fails the request. It closes over the same emit/traceId/mode locals as the compression.completed emit right below it (line 1749); the reusable per-engine emission lives in strategySelector.ts (reportEngineStep + StackedCompressionStep) and the studio reducers in compressionFlowModel.ts (both <cap). Not extractable without hiding the emit boundary, mirroring the prior compression rebaselines (#4210/#4004). Structural shrink of chatCore.ts tracked in #3501.",
|
||||
"_rebaseline_2026_06_18_4210_engine_breakdown": "PR #4210 own growth: chatCore.ts 5060->5063 (+3 = wire ensureEngineBreakdown(result.stats) into the existing compression.completed emit + its import line). Single-engine modes (rtk/lite/standard/aggressive/ultra) leave stats.engineBreakdown empty, which made the dashboard studio render an empty Input->Output pipeline (no engine node); the synthesized 1-entry breakdown lives in the new pure leaf open-sse/services/compression/engineBreakdown.ts (<cap), mirroring seedLatestCompressionRunFromDb. The +3 is the import + a 2-line explanatory comment at the emit chokepoint; not extractable further without hiding the emit boundary.",
|
||||
@@ -70,7 +71,7 @@
|
||||
"open-sse/executors/muse-spark-web.ts": 1284,
|
||||
"open-sse/executors/perplexity-web.ts": 1013,
|
||||
"open-sse/handlers/audioSpeech.ts": 965,
|
||||
"open-sse/handlers/chatCore.ts": 5095,
|
||||
"open-sse/handlers/chatCore.ts": 5102,
|
||||
"open-sse/handlers/imageGeneration.ts": 3777,
|
||||
"open-sse/handlers/responseSanitizer.ts": 1103,
|
||||
"open-sse/handlers/search.ts": 1546,
|
||||
|
||||
@@ -16,7 +16,13 @@ export const anthropicProvider: RegistryEntry = {
|
||||
"Anthropic-Beta": ANTHROPIC_BETA_API_KEY,
|
||||
},
|
||||
models: [
|
||||
{ id: "claude-opus-4.7", name: "Claude Opus 4.7" },
|
||||
{
|
||||
id: "claude-opus-4.7",
|
||||
name: "Claude Opus 4.7",
|
||||
// Opus 4.7+ rejects non-default temperature/top_p/top_k with a 400 (sampling fixed;
|
||||
// reasoning via output_config.effort). Mirrors the dashed `claude` registry ids.
|
||||
unsupportedParams: ["temperature", "top_p", "top_k"],
|
||||
},
|
||||
{ id: "claude-opus-4.6", name: "Claude Opus 4.6" },
|
||||
{ id: "claude-sonnet-4.6", name: "Claude Sonnet 4.6" },
|
||||
{ id: "claude-sonnet-4.5", name: "Claude Sonnet 4.6" },
|
||||
|
||||
@@ -33,18 +33,23 @@ export const claudeProvider: RegistryEntry = {
|
||||
name: "Claude Fable 5",
|
||||
contextLength: 1000000,
|
||||
maxOutputTokens: 128000,
|
||||
// Opus 4.7+/Fable 5 reject non-default temperature/top_p/top_k with a 400 (sampling
|
||||
// is fixed; reasoning is steered by output_config.effort). Strip them before dispatch.
|
||||
unsupportedParams: ["temperature", "top_p", "top_k"],
|
||||
},
|
||||
{
|
||||
id: "claude-opus-4-8",
|
||||
name: "Claude Opus 4.8",
|
||||
contextLength: 1000000,
|
||||
maxOutputTokens: 128000,
|
||||
unsupportedParams: ["temperature", "top_p", "top_k"],
|
||||
},
|
||||
{
|
||||
id: "claude-opus-4-7",
|
||||
name: "Claude Opus 4.7",
|
||||
contextLength: 1000000,
|
||||
maxOutputTokens: 128000,
|
||||
unsupportedParams: ["temperature", "top_p", "top_k"],
|
||||
},
|
||||
{
|
||||
id: "claude-opus-4-6",
|
||||
|
||||
@@ -80,6 +80,7 @@ import {
|
||||
} from "../services/modelStrip.ts";
|
||||
import { resolveModelAlias } from "../services/modelDeprecation.ts";
|
||||
import { normalizeMimoThinking } from "../services/mimoThinking.ts";
|
||||
import { normalizeClaudeAdaptiveThinking } from "../services/claudeAdaptiveThinking.ts";
|
||||
import { getUnsupportedParams } from "../config/providerRegistry.ts";
|
||||
import { supportsMaxTokens } from "@/lib/modelCapabilities.ts";
|
||||
import { normalizeThinkingForModel } from "@/shared/constants/modelSpecs.ts";
|
||||
@@ -2535,6 +2536,12 @@ export async function handleChatCore({
|
||||
// when the resolved target model rejects it; models that accept `disabled` are untouched.
|
||||
if (typeof finalModelToUpstream === "string") {
|
||||
translatedBody = normalizeThinkingForModel(translatedBody, finalModelToUpstream);
|
||||
// Claude Opus 4.7+/Fable 5 removed manual extended thinking: `thinking.type:"enabled"`
|
||||
// or any `thinking.budget_tokens` is a hard 400. Collapse any manual thinking that
|
||||
// reached this point (passthrough legacy shape, reasoning_effort buckets, per-model
|
||||
// defaults) to `{type:"adaptive"}` — effort stays on `output_config.effort`. Keyed on
|
||||
// the resolved upstream model, so it covers every routing mode. See claudeAdaptiveThinking.ts.
|
||||
translatedBody = normalizeClaudeAdaptiveThinking(translatedBody, finalModelToUpstream);
|
||||
}
|
||||
|
||||
// Xiaomi MiMo controls reasoning ONLY via `thinking:{type:"enabled"|"disabled"}` and
|
||||
|
||||
52
open-sse/services/claudeAdaptiveThinking.ts
Normal file
52
open-sse/services/claudeAdaptiveThinking.ts
Normal file
@@ -0,0 +1,52 @@
|
||||
import { isAdaptiveThinkingOnly } from "@/shared/constants/modelSpecs.ts";
|
||||
|
||||
type JsonRecord = Record<string, unknown>;
|
||||
|
||||
function asRecord(value: unknown): JsonRecord | null {
|
||||
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Collapse manual extended thinking to adaptive for Claude models that no longer accept it.
|
||||
*
|
||||
* Claude Opus 4.7 and later (Opus 4.7/4.8, Fable 5) removed manual extended thinking: the
|
||||
* Messages API returns HTTP 400 for `thinking.type:"enabled"` and for ANY
|
||||
* `thinking.budget_tokens`. Reasoning is steered exclusively by `output_config.effort`
|
||||
* (Anthropic migration guide, 2026-05-19). OmniRoute can still produce a manual thinking
|
||||
* block on these models from several paths — a Claude-native passthrough client sending the
|
||||
* legacy shape, the OpenAI→Claude translator's reasoning_effort buckets, or a per-model
|
||||
* thinking default — so this is the final, provider-agnostic guard keyed on the target model.
|
||||
*
|
||||
* Returns a NEW object only when it changes the body:
|
||||
* - `thinking.type:"enabled"` → `"adaptive"` (the only supported mode);
|
||||
* - `thinking.budget_tokens` / `thinking.max_tokens` → dropped (rejected extras).
|
||||
* `thinking.type:"adaptive"` is left as-is (just stripped of any stray budget), and
|
||||
* `thinking.type:"disabled"` is left untouched — that's handled separately by
|
||||
* `normalizeThinkingForModel` for the models that reject `disabled` (#3554).
|
||||
*
|
||||
* No-op (returns the same reference) when the model is not adaptive-only, when there is no
|
||||
* thinking object, or when the thinking object already carries no manual-budget signal —
|
||||
* so adaptive defaults and effort hints reach the model unchanged.
|
||||
*/
|
||||
export function normalizeClaudeAdaptiveThinking<T extends Record<string, unknown>>(
|
||||
body: T,
|
||||
model: string | null | undefined
|
||||
): T {
|
||||
if (!isAdaptiveThinkingOnly(model)) return body;
|
||||
const record = asRecord(body);
|
||||
if (!record) return body;
|
||||
|
||||
const thinking = asRecord(record.thinking);
|
||||
if (!thinking) return body;
|
||||
|
||||
const isManualType = thinking.type === "enabled";
|
||||
const hasBudget = thinking.budget_tokens !== undefined || thinking.max_tokens !== undefined;
|
||||
if (!isManualType && !hasBudget) return body;
|
||||
|
||||
const nextThinking: JsonRecord = { ...thinking };
|
||||
if (nextThinking.type === "enabled") nextThinking.type = "adaptive";
|
||||
delete nextThinking.budget_tokens;
|
||||
delete nextThinking.max_tokens;
|
||||
|
||||
return { ...record, thinking: nextThinking } as T;
|
||||
}
|
||||
@@ -6,6 +6,11 @@ import { adjustMaxTokens } from "../helpers/maxTokensHelper.ts";
|
||||
import { sanitizeToolId } from "../helpers/schemaCoercion.ts";
|
||||
import { DEFAULT_THINKING_CLAUDE_SIGNATURE } from "../../config/defaultThinkingSignature.ts";
|
||||
import { capMaxOutputTokens } from "../../../src/lib/modelCapabilities.ts";
|
||||
import { isAdaptiveThinkingOnly } from "../../../src/shared/constants/modelSpecs.ts";
|
||||
|
||||
// Reasoning-effort levels Anthropic accepts on `output_config.effort`. Used to steer
|
||||
// adaptive-only Claude models (Opus 4.7+/Fable 5) without ever emitting a manual budget.
|
||||
const ADAPTIVE_EFFORT_LEVELS = new Set(["low", "medium", "high", "xhigh", "max"]);
|
||||
|
||||
// Prefix for Claude OAuth tool names to avoid conflicts
|
||||
// Can be disabled per-request via body._disableToolPrefix = true
|
||||
@@ -458,7 +463,22 @@ export function openaiToClaudeRequest(model, body, stream) {
|
||||
: requestedEffort === "xhigh" && !supportsXHighEffort("claude", model)
|
||||
? "high"
|
||||
: requestedEffort;
|
||||
if (normalizedEffort === "max" || normalizedEffort === "xhigh") {
|
||||
if (isAdaptiveThinkingOnly(model)) {
|
||||
// Opus 4.7+/Fable 5 removed manual extended thinking: a fixed `budget_tokens`
|
||||
// (or `type:"enabled"`) is a hard 400. Steer EVERY level via adaptive +
|
||||
// output_config.effort instead of the budget buckets below. Unrecognized levels
|
||||
// leave thinking unset so the model keeps its adaptive default rather than 400ing
|
||||
// on an invalid effort value.
|
||||
if (ADAPTIVE_EFFORT_LEVELS.has(normalizedEffort)) {
|
||||
result.thinking = {
|
||||
type: "adaptive",
|
||||
};
|
||||
result.output_config = {
|
||||
...(result.output_config || {}),
|
||||
effort: normalizedEffort,
|
||||
};
|
||||
}
|
||||
} else if (normalizedEffort === "max" || normalizedEffort === "xhigh") {
|
||||
result.thinking = {
|
||||
type: "adaptive",
|
||||
};
|
||||
|
||||
@@ -19,6 +19,12 @@ export interface ModelSpec {
|
||||
// (upstream returns 400). Used to normalize the request when a combo/route substitutes
|
||||
// this model after the client already chose `disabled`. See issue #3554.
|
||||
rejectsThinkingDisabled?: boolean;
|
||||
// Model ONLY supports adaptive thinking: manual extended thinking was removed. Sending
|
||||
// `thinking.type:"enabled"` or any `thinking.budget_tokens` returns HTTP 400; reasoning
|
||||
// is steered exclusively by `output_config.effort` (low/medium/high/xhigh/max). True for
|
||||
// Claude Opus 4.7 and later (Opus 4.7/4.8, Fable 5). Per Anthropic's migration guide
|
||||
// (2026-05-19): "Any request that tries to set a fixed thinking budget gets a 400 error."
|
||||
adaptiveThinkingOnly?: boolean;
|
||||
// Explicit operator override for the no-thinking gateway alias (Fase 8.1). When unset,
|
||||
// the catalog auto-advertises a `claude-3-omniroute-no-thinking/…` variant for
|
||||
// Claude-family thinking-capable models that honor `disabled`. Set `true` to force the
|
||||
@@ -211,16 +217,17 @@ export const MODEL_SPECS: Record<string, ModelSpec> = {
|
||||
"claude-opus-4-7": {
|
||||
maxOutputTokens: 128000,
|
||||
contextWindow: 1000000,
|
||||
// Anthropic accepts thinking.budget_tokens in [1024, 128000]; cap
|
||||
// a bit below to leave headroom for the visible response within
|
||||
// max_tokens. Without this cap, adaptive scaling on top of an
|
||||
// `output_config.effort=max` request can push past 128000 and
|
||||
// trigger a 400 "budget out of range" from Anthropic.
|
||||
// Opus 4.7 removed manual extended thinking: a fixed `thinking.budget_tokens`
|
||||
// (or `thinking.type:"enabled"`) returns 400. Reasoning is adaptive-only and
|
||||
// steered by `output_config.effort`. defaultThinkingBudget/thinkingBudgetCap
|
||||
// are retained only as caps for any legacy budget path; the request flow
|
||||
// collapses manual thinking to adaptive before dispatch (see adaptiveThinkingOnly).
|
||||
defaultThinkingBudget: 32000,
|
||||
thinkingBudgetCap: 120000,
|
||||
supportsThinking: true,
|
||||
supportsTools: true,
|
||||
supportsVision: true,
|
||||
adaptiveThinkingOnly: true,
|
||||
aliases: BEDROCK_CLAUDE_ALIASES("claude-opus-4-7", "claude-opus-4.7"),
|
||||
},
|
||||
|
||||
@@ -235,6 +242,8 @@ export const MODEL_SPECS: Record<string, ModelSpec> = {
|
||||
supportsVision: true,
|
||||
// Fable 5 defaults to adaptive thinking and rejects `thinking.type:"disabled"` (#3554).
|
||||
rejectsThinkingDisabled: true,
|
||||
// …and, like Opus 4.7+, rejects manual budgets/`type:"enabled"` (adaptive-only).
|
||||
adaptiveThinkingOnly: true,
|
||||
aliases: BEDROCK_CLAUDE_ALIASES("claude-fable-5"),
|
||||
},
|
||||
|
||||
@@ -249,6 +258,7 @@ export const MODEL_SPECS: Record<string, ModelSpec> = {
|
||||
supportsThinking: true,
|
||||
supportsTools: true,
|
||||
supportsVision: true,
|
||||
adaptiveThinkingOnly: true,
|
||||
aliases: BEDROCK_CLAUDE_ALIASES("claude-opus-4-8", "claude-opus-4.8"),
|
||||
},
|
||||
|
||||
@@ -515,6 +525,18 @@ export function getDefaultThinkingBudget(modelId: string): number {
|
||||
return getModelSpec(modelId)?.defaultThinkingBudget ?? 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* True when the resolved model only supports adaptive thinking and rejects manual
|
||||
* extended thinking. For these models (Claude Opus 4.7+/Fable 5) a `thinking.type:"enabled"`
|
||||
* or any `thinking.budget_tokens` is a hard 400 — reasoning must be steered via
|
||||
* `output_config.effort`. Used by the request flow to collapse manual thinking to
|
||||
* `{type:"adaptive"}` before dispatch. Matches dated/Bedrock aliases via getModelSpec.
|
||||
*/
|
||||
export function isAdaptiveThinkingOnly(modelId: string | null | undefined): boolean {
|
||||
if (typeof modelId !== "string" || modelId.length === 0) return false;
|
||||
return getModelSpec(modelId)?.adaptiveThinkingOnly === true;
|
||||
}
|
||||
|
||||
export function capThinkingBudget(modelId: string, budget: number): number {
|
||||
const cap = getModelSpec(modelId)?.thinkingBudgetCap ?? budget;
|
||||
return Math.min(budget, cap);
|
||||
|
||||
72
tests/unit/claude-adaptive-sampling-params.test.ts
Normal file
72
tests/unit/claude-adaptive-sampling-params.test.ts
Normal file
@@ -0,0 +1,72 @@
|
||||
/**
|
||||
* Claude Opus 4.7+/Fable 5 sampling-param strip + adaptive-only flag.
|
||||
*
|
||||
* Anthropic's Opus 4.7+ generation rejects non-default `temperature`/`top_p`/`top_k` with a
|
||||
* 400 (sampling is fixed; reasoning is steered by output_config.effort). These tests pin both
|
||||
* the registry `unsupportedParams` that drive the strip at the chatCore dispatch point and
|
||||
* the `isAdaptiveThinkingOnly` model flag — with regression guards that pre-4.7 models keep
|
||||
* accepting sampling params and manual thinking.
|
||||
*/
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { getUnsupportedParams } from "../../open-sse/config/providerRegistry.ts";
|
||||
import { isAdaptiveThinkingOnly } from "../../src/shared/constants/modelSpecs.ts";
|
||||
|
||||
const SAMPLING = ["temperature", "top_p", "top_k"];
|
||||
|
||||
test("claude registry strips temperature/top_p/top_k for Opus 4.7+/Fable 5", () => {
|
||||
for (const model of ["claude-opus-4-8", "claude-opus-4-7", "claude-fable-5"]) {
|
||||
const unsupported = getUnsupportedParams("claude", model);
|
||||
for (const param of SAMPLING) {
|
||||
assert.ok(
|
||||
unsupported.includes(param),
|
||||
`${model} must list ${param} as unsupported (400 on Anthropic otherwise)`
|
||||
);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test("anthropic registry (dotted ids) strips sampling params for Opus 4.7", () => {
|
||||
const unsupported = getUnsupportedParams("anthropic", "claude-opus-4.7");
|
||||
for (const param of SAMPLING) {
|
||||
assert.ok(unsupported.includes(param), `claude-opus-4.7 must list ${param} as unsupported`);
|
||||
}
|
||||
});
|
||||
|
||||
test("pre-4.7 Claude models still accept sampling params (regression guard)", () => {
|
||||
for (const [provider, model] of [
|
||||
["claude", "claude-opus-4-6"],
|
||||
["claude", "claude-opus-4-5-20251101"],
|
||||
["claude", "claude-sonnet-4-5-20250929"],
|
||||
["claude", "claude-haiku-4-5-20251001"],
|
||||
["anthropic", "claude-opus-4.6"],
|
||||
] as const) {
|
||||
const unsupported = getUnsupportedParams(provider, model);
|
||||
for (const param of SAMPLING) {
|
||||
assert.ok(
|
||||
!unsupported.includes(param),
|
||||
`${provider}/${model} must NOT strip ${param} — it still accepts sampling`
|
||||
);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test("isAdaptiveThinkingOnly is true only for Opus 4.7+/Fable 5", () => {
|
||||
for (const model of ["claude-opus-4-8", "claude-opus-4-7", "claude-fable-5"]) {
|
||||
assert.equal(isAdaptiveThinkingOnly(model), true, `${model} is adaptive-only`);
|
||||
}
|
||||
for (const model of [
|
||||
"claude-opus-4-6",
|
||||
"claude-opus-4-5-20251101",
|
||||
"claude-sonnet-4-6",
|
||||
"claude-haiku-4-5-20251001",
|
||||
]) {
|
||||
assert.equal(isAdaptiveThinkingOnly(model), false, `${model} still supports manual thinking`);
|
||||
}
|
||||
assert.equal(isAdaptiveThinkingOnly(null), false);
|
||||
assert.equal(isAdaptiveThinkingOnly(""), false);
|
||||
});
|
||||
|
||||
test("isAdaptiveThinkingOnly resolves Bedrock/dated aliases", () => {
|
||||
assert.equal(isAdaptiveThinkingOnly("anthropic.claude-opus-4-8"), true);
|
||||
});
|
||||
105
tests/unit/claude-adaptive-thinking-normalize.test.ts
Normal file
105
tests/unit/claude-adaptive-thinking-normalize.test.ts
Normal file
@@ -0,0 +1,105 @@
|
||||
/**
|
||||
* Claude adaptive-thinking normalization — `normalizeClaudeAdaptiveThinking`.
|
||||
*
|
||||
* Claude Opus 4.7+/Fable 5 removed manual extended thinking: `thinking.type:"enabled"` and
|
||||
* any `thinking.budget_tokens` return HTTP 400 (Anthropic migration guide, 2026-05-19).
|
||||
* These tests pin the final guard that collapses any manual thinking that reached the
|
||||
* dispatch point to `{type:"adaptive"}`, while leaving non-adaptive-only models and
|
||||
* already-adaptive bodies untouched.
|
||||
*/
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { normalizeClaudeAdaptiveThinking } from "../../open-sse/services/claudeAdaptiveThinking.ts";
|
||||
|
||||
test("manual thinking:{type:'enabled', budget_tokens} → adaptive, budget dropped (Opus 4.8)", () => {
|
||||
const body = {
|
||||
model: "claude-opus-4-8",
|
||||
messages: [],
|
||||
thinking: { type: "enabled", budget_tokens: 131072 },
|
||||
};
|
||||
const result = normalizeClaudeAdaptiveThinking(body, "claude-opus-4-8");
|
||||
assert.deepEqual(result.thinking, { type: "adaptive" });
|
||||
});
|
||||
|
||||
test("Claude-shaped thinking:{type:'enabled', max_tokens} → adaptive, max_tokens dropped", () => {
|
||||
const body = {
|
||||
model: "claude-opus-4-7",
|
||||
messages: [],
|
||||
thinking: { type: "enabled", budget_tokens: 4096, max_tokens: 8000 },
|
||||
};
|
||||
const result = normalizeClaudeAdaptiveThinking(body, "claude-opus-4-7");
|
||||
assert.deepEqual(result.thinking, { type: "adaptive" });
|
||||
});
|
||||
|
||||
test("type:'enabled' with no budget still flips to adaptive (manual mode is gone)", () => {
|
||||
const body = { model: "claude-fable-5", messages: [], thinking: { type: "enabled" } };
|
||||
const result = normalizeClaudeAdaptiveThinking(body, "claude-fable-5");
|
||||
assert.deepEqual(result.thinking, { type: "adaptive" });
|
||||
});
|
||||
|
||||
test("thinking:{type:'adaptive'} is returned UNTOUCHED (same reference)", () => {
|
||||
const body = { model: "claude-opus-4-8", messages: [], thinking: { type: "adaptive" } };
|
||||
const result = normalizeClaudeAdaptiveThinking(body, "claude-opus-4-8");
|
||||
assert.equal(result, body, "already-adaptive body must not be reallocated");
|
||||
});
|
||||
|
||||
test("adaptive thinking carrying a stray budget_tokens has it stripped", () => {
|
||||
const body = {
|
||||
model: "claude-opus-4-8",
|
||||
messages: [],
|
||||
thinking: { type: "adaptive", budget_tokens: 5 },
|
||||
};
|
||||
const result = normalizeClaudeAdaptiveThinking(body, "claude-opus-4-8");
|
||||
assert.deepEqual(result.thinking, { type: "adaptive" });
|
||||
});
|
||||
|
||||
test("thinking:{type:'disabled'} is left untouched (handled by normalizeThinkingForModel)", () => {
|
||||
const body = { model: "claude-opus-4-8", messages: [], thinking: { type: "disabled" } };
|
||||
const result = normalizeClaudeAdaptiveThinking(body, "claude-opus-4-8");
|
||||
assert.equal(result, body, "disabled is a separate concern; do not touch it here");
|
||||
});
|
||||
|
||||
test("NON-adaptive-only model keeps its manual budget (regression guard for Opus 4.6)", () => {
|
||||
const body = {
|
||||
model: "claude-opus-4-6",
|
||||
messages: [],
|
||||
thinking: { type: "enabled", budget_tokens: 96000 },
|
||||
};
|
||||
const result = normalizeClaudeAdaptiveThinking(body, "claude-opus-4-6");
|
||||
assert.equal(result, body, "Opus 4.6 still supports manual extended thinking");
|
||||
assert.deepEqual(result.thinking, { type: "enabled", budget_tokens: 96000 });
|
||||
});
|
||||
|
||||
test("body without a thinking object is returned UNTOUCHED (same reference)", () => {
|
||||
const body = { model: "claude-opus-4-8", messages: [{ role: "user", content: "hi" }] };
|
||||
const result = normalizeClaudeAdaptiveThinking(body, "claude-opus-4-8");
|
||||
assert.equal(result, body);
|
||||
});
|
||||
|
||||
test("non-object body / empty model are returned unchanged", () => {
|
||||
const body = null as unknown as Record<string, unknown>;
|
||||
assert.equal(normalizeClaudeAdaptiveThinking(body, "claude-opus-4-8"), body);
|
||||
const ok = { model: "claude-opus-4-8", thinking: { type: "enabled" } };
|
||||
assert.equal(normalizeClaudeAdaptiveThinking(ok, ""), ok, "empty model → no-op");
|
||||
});
|
||||
|
||||
test("Bedrock/dated alias still resolves the adaptive-only spec", () => {
|
||||
const body = { thinking: { type: "enabled", budget_tokens: 1000 } };
|
||||
// BEDROCK_CLAUDE_ALIASES generates `anthropic.claude-opus-4-8` etc.; getModelSpec resolves it.
|
||||
const result = normalizeClaudeAdaptiveThinking(body, "anthropic.claude-opus-4-8");
|
||||
assert.deepEqual(result.thinking, { type: "adaptive" });
|
||||
});
|
||||
|
||||
test("unrelated fields are preserved when collapsing thinking", () => {
|
||||
const body = {
|
||||
model: "claude-opus-4-8",
|
||||
messages: [{ role: "user", content: "hi" }],
|
||||
output_config: { effort: "high" },
|
||||
thinking: { type: "enabled", budget_tokens: 32000 },
|
||||
};
|
||||
const result = normalizeClaudeAdaptiveThinking(body, "claude-opus-4-8") as Record<string, unknown>;
|
||||
assert.equal(result.model, "claude-opus-4-8");
|
||||
assert.deepEqual(result.messages, [{ role: "user", content: "hi" }]);
|
||||
assert.deepEqual(result.output_config, { effort: "high" });
|
||||
assert.deepEqual(result.thinking, { type: "adaptive" });
|
||||
});
|
||||
@@ -349,15 +349,17 @@ test("OpenAI -> Claude preserves max effort except for Haiku models", () => {
|
||||
assert.equal(haiku.max_tokens, 64000);
|
||||
});
|
||||
|
||||
test("OpenAI -> Claude fits thinking budget within Opus 4.7 output cap (regression)", () => {
|
||||
test("OpenAI -> Claude fits thinking budget within a 128k output cap (regression)", () => {
|
||||
// Real-world OpenCode scenario: caller asks for max_tokens=32000 with high effort.
|
||||
// High effort maps to budget=131072. The previous naive
|
||||
// `budget + 8192 = 139264` exceeded Opus 4.7's 128000 output cap and caused
|
||||
// `budget + 8192 = 139264` exceeded the 128000 output cap and caused
|
||||
// HTTP 400 "max_tokens > 128000".
|
||||
// fitThinkingToMaxTokens must preserve caller's 32000 response room and
|
||||
// shrink budget to (128000 - 32000) = 96000.
|
||||
// Pinned on Opus 4.6 — a model that still uses manual budgets. Opus 4.7+/Fable 5 are
|
||||
// adaptive-only now (no budget_tokens), so their effort path is covered separately below.
|
||||
const result = openaiToClaudeRequest(
|
||||
"claude-opus-4-7",
|
||||
"claude-opus-4-6",
|
||||
{
|
||||
messages: [{ role: "user", content: "Reason about something hard" }],
|
||||
max_tokens: 32000,
|
||||
@@ -376,6 +378,56 @@ test("OpenAI -> Claude fits thinking budget within Opus 4.7 output cap (regressi
|
||||
);
|
||||
});
|
||||
|
||||
test("OpenAI -> Claude steers adaptive-only models via output_config.effort for EVERY level", () => {
|
||||
// Opus 4.7+/Fable 5 reject a manual `thinking.budget_tokens`/`type:"enabled"` with 400.
|
||||
// reasoning_effort low/medium/high must therefore map to adaptive + output_config.effort
|
||||
// (preserving the requested level), NOT to the budget buckets older models use.
|
||||
for (const effort of ["low", "medium", "high", "xhigh", "max"]) {
|
||||
for (const model of ["claude-opus-4-8", "claude-opus-4-7", "claude-fable-5"]) {
|
||||
const result = openaiToClaudeRequest(
|
||||
model,
|
||||
{
|
||||
messages: [{ role: "user", content: "Reason" }],
|
||||
max_tokens: 4000,
|
||||
reasoning_effort: effort,
|
||||
},
|
||||
false
|
||||
);
|
||||
assert.deepEqual(
|
||||
result.thinking,
|
||||
{ type: "adaptive" },
|
||||
`${model} @ ${effort} must use adaptive thinking, never a manual budget`
|
||||
);
|
||||
assert.deepEqual(
|
||||
result.output_config,
|
||||
{ effort },
|
||||
`${model} @ ${effort} must carry the effort on output_config`
|
||||
);
|
||||
assert.equal(
|
||||
(result.thinking as Record<string, unknown>).budget_tokens,
|
||||
undefined,
|
||||
`${model} @ ${effort} must NOT emit budget_tokens (hard 400 on adaptive-only models)`
|
||||
);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test("OpenAI -> Claude keeps manual budgets for low/medium/high on pre-4.7 models (regression)", () => {
|
||||
// Opus 4.6 still supports manual extended thinking: the budget buckets must be untouched.
|
||||
const result = openaiToClaudeRequest(
|
||||
"claude-opus-4-6",
|
||||
{
|
||||
messages: [{ role: "user", content: "Reason" }],
|
||||
max_tokens: 20000,
|
||||
reasoning_effort: "medium",
|
||||
},
|
||||
false
|
||||
);
|
||||
assert.equal((result.thinking as { type: string }).type, "enabled");
|
||||
assert.equal((result.thinking as { budget_tokens: number }).budget_tokens, 10240);
|
||||
assert.equal(result.output_config, undefined);
|
||||
});
|
||||
|
||||
test("OpenAI -> Claude can disable OAuth prefixes and Antigravity strips Claude-only prompting", () => {
|
||||
const baseBody = {
|
||||
messages: [
|
||||
|
||||
Reference in New Issue
Block a user