pjandro/OmniRoute
1
0
Fork 0
mirror of https://github.com/diegosouzapw/OmniRoute.git synced 2026-09-21 14:22:14 +03:00
Code Issues Packages Projects Releases Wiki Activity

fix(chat): preserve suffix reasoning intent across model attempts (#13720)

Browse Source 
* chat/suffix-effort: keep reasoning intent tied to each model attempt

Carry resolved suffix effort through dispatch without treating a derived
value as explicit client input. Prepare reasoning defaults and dependent
parameter constraints for each handler attempt so a replacement model
does not inherit the original model's suffix.

Keep explicit reasoning choices in context-aware request hashes to avoid
sharing concurrent responses across different effort settings. Preserve
the legacy hash interface and tenant namespace.

Exercise retries, credential refresh, tool follow-ups, replacement models
and overlapping requests with local HTTP and targeted regression tests.

Signed-off-by: Minxi Hou <houminxi@gmail.com>

* chat/upstream-body: separate normalization from async payload preparation

Keep synchronous per-attempt normalization together so payload preparation
stays within the function size and complexity limits without changing its
ordering or explicit reasoning semantics. Condense redundant provider
selection comments to retain the formatted file within its size ceiling.

Signed-off-by: Minxi Hou <houminxi@gmail.com>

* changelog: record the suffix-effort propagation fix

Signed-off-by: Minxi Hou <houminxi@gmail.com>

* fix(quality): rebaseline file-size cap for chatHelpers.ts growth

The release tip independently grew src/sse/handlers/chatHelpers.ts from
1164 to 1213 lines while the frozen cap sat at 1214; this PR's own +3
lines (threading resolvedThinkingEffort through resolveModelOrError and
executeChatWithBreaker) push the merged result to 1217, past the cap.
Owner-approved exception for this file only, with the measured growth
breakdown recorded in the baseline entry.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Signed-off-by: Minxi Hou <houminxi@gmail.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
This commit is contained in:
Bob.Hou
2026-09-17 17:52:59 -04:00
committed by GitHub
parent 7d69b02a29
commit b9dd80c8c2
10 changed files with 1251 additions and 165 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
changelog.d/fixes/13720-suffix-effort-propagation.md Normal file
View File

@@ -0,0 +1 @@
- fix(chat): preserve suffix-model reasoning effort across model attempts so a replacement model no longer inherits or drops the original suffix, and keep explicit reasoning choices in request dedup hashes (#13720)

4
config/quality/file-size-baseline.json
View File

@@ -358,8 +358,10 @@
"_rebaseline_2026_07_27_3850_relax_filesize_cap": "OWNER-APPROVED TEMPORARY relax for v3.8.50-3.8.54 PREPARE phase (docs/ROADMAP.md). cap 800->900 (+100), testCap 800->900 (+100). Targets: decompose-existing-frozen unchanged (frozen still only-shrink); this only relaxes the cap for NEW files in the decompose/extract-while-PREPARE phase (.51='executor registry in-place' and .52='combo.ts decomposition' create new leaf modules above 800). RE-TIGHTENING MANDATORY in v3.8.51: cap target 850 = 850 once decomposition wave stabilizes. SUPERSEDED by _rebaseline_2026_07_27_3850_relax_filesize_cap_v2_20pct (v1 +20% buffer) — retained for audit. Tracked via same roadmap issue.",
"_rebaseline_2026_07_27_v3849_train1h": "Merge-train 1H (31 PRs) — owner-approved 2026-07-27. Two distinct causes, kept separate on purpose: (1) GENUINE irreducible growth at existing chokepoints — providerLimits/auth (#8632 Kimi quota-reset recovery), rateLimitManager (#8616 idle wedged limiters), models-catalog-route.test (#8610 OpenCode Go effort aliases); (2) COLLISION with #8585, which banked shrinks measured on the pre-train release tip while 30 sibling PRs in the SAME train grew those files again — chat/accountFallback (#8628), chatCore (#8613), videoGeneration (#8581), imageGeneration. The zero-headroom frozen entries cannot absorb either. Ceilings re-pinned to the post-merge tip; #8612 (also in this train) automates shrink-banking so this self-inflicted drift stops recurring. Detail: src/lib/usage/providerLimits.ts 1006->1013 (#8632); src/sse/services/auth.ts 2492->2508 (#8632); open-sse/services/rateLimitManager.ts 1014->1060 (#8616); src/sse/handlers/chat.ts 1842->1845 (#8628); open-sse/handlers/chatCore.ts 4939->4955 (#8613); open-sse/handlers/imageGeneration.ts 3100->3101 ((sem PR — teto do #8585)); open-sse/handlers/videoGeneration.ts 1038->1063 (#8581); open-sse/services/accountFallback.ts 1965->1966 (#8628); tests/unit/models-catalog-route.test.ts 1608->1636 (#8610)",
"frozen": {
"src/sse/handlers/chatHelpers.ts": 1253,
"_rebaseline_2026_09_17_13720_merge_release_v3851": "Merge de release/v3.8.51 na #13720 (2026-09-17). src/sse/handlers/chatHelpers.ts 1246 -> 1253, decomposto: 1246 -> 1250 e crescimento INHERITED do tip (base-red ja presente em origin/release/v3.8.51 no commit 9688032451fc, arquivo com 1250 linhas contra cap 1246 — nao e desta PR e nao foi introduzido por este merge); 1250 -> 1253 sao as MESMAS +3 linhas da propria #13720 ja auditadas e aprovadas pelo dono na entrada _rebaseline_2026_09_16_13720_suffix_effort_propagation abaixo (threading de resolvedThinkingEffort). Nenhum outro teto foi tocado por este merge; tests/unit/chatcore-translation-paths.test.ts (3449 > 3447) permanece vermelho de proposito — e base-red herdado e a PR nao toca o arquivo.",
"_rebaseline_2026_09_16_13720_suffix_effort_propagation": "OWNER-APPROVED 2026-09-16 (explicit exception for this unit only, chatHelpers.ts only). PR #13720 (HouMinXi, suffix-effort propagation across model attempts): src/sse/handlers/chatHelpers.ts merge-base (before PR's own commit) was 1164; the release tip independently grew it to 1213 (+49, unrelated merged PRs) while the frozen cap sat at 1214 to cover exactly that tip growth. The PR's own diff on this file is +3 lines only (threading resolvedThinkingEffort: one field on resolveModelOrError's return object, one destructured param and one passthrough call-site argument in executeChatWithBreaker — see commit 4fdb0c5851b7f645efbe5627cd0babb0c3d230c3), taking the merged result to 1216 (1217 per check-file-size.mjs's countLines, which counts the trailing newline as an extra split segment). All 3 added lines are single-property additions inside existing multi-line object literals/signatures; there is no redundant or duplicated line in the PR's own hunks to trim, and none of the +3 lines are outside the PR's own diff. Covered by tests/unit/suffix-effort-propagation.test.ts (27/27), tests/unit/chatcore-upstream-body.test.ts + tests/unit/request-dedup-tenant-isolation.test.ts (57/57), all green against this exact head.",
"_rebaseline_2026_09_17_13947_tip_growth": "Base-red drain da PR #13947 (Refs #13866) — crescimento de PRODUCAO que chegou pelo tip e nunca foi rebaselinado; nenhum destes arquivos e tocado por esta PR. #12906 (d70f43d4, retry empty_response 502 + timeout de inicio de resposta ciente de reasoning): src/sse/handlers/chat.ts 2498->2500, src/sse/handlers/chatHelpers.ts 1231->1245, open-sse/utils/proxyFetch.ts 1275->1276, open-sse/utils/stream.ts 3098->3123. #12904 (f3acf4f8, injecao unica do system prompt global pos-traducao) + #12910 (051576fd, finalizacao de cache semantico por request id exato): open-sse/handlers/chatCore.ts 6181->6203. Anteriores ao lote, ja acima do cap na base 3d5baf13: open-sse/handlers/imageGeneration.ts 3293->3304 (#13748, b97338a8) e open-sse/services/combo/roundRobinCombo.ts 1213->1221 (#13776, aeba6b1a). Registrado contra o estado mergeado; nenhum outro cap e tocado.",
"src/sse/handlers/chatHelpers.ts": 1246,
"_rebaseline_2026_09_15_13609_mistral_ambiguous_401": "PR #13609 rework (maxmad64bis, bare Mistral 401 soft lockout behind MISTRAL_AMBIGUOUS_401_SOFT_LOCKOUT, default off). open-sse/services/accountFallback.ts 2469->2501 (+32): +14 are the change itself (shared-predicate + flag imports, the documented ambiguousAuth field on the checkFallbackError return type, and the flag-gated 401 branch formatted normally instead of the PR's 139-char squeezed configuredRule line); +18 are the lint-staged prettier pass normalizing lines that were already unformatted on the release tip (multi-import, ISO_RETRY_RE, two regex arrays, persistAntigravityFamilyCooldownIfQuota call, applyErrorState guard, trailing commas) — pure formatting, no logic. src/sse/services/auth.ts 3556->3557 (+1): markAccountUnavailable passes connectionId to resolveTerminalConnectionStatus so the soft-strike bound is per connection. The predicate and strike tracker live in the leaf open-sse/services/accountFallback/mistralAmbiguousAuth.ts (under cap). Covered by tests/unit/provider-401-ambiguous-runtime.test.ts (flag off/on, end-to-end through markAccountUnavailable).",
"_rebaseline_2026_06_22_4644_deepseek_web_tools": "PR #4644 (BugsBag/robust deepseek-web tool-call parsing): open-sse/executors/deepseek-web.ts 1117->1125 (+8). The new agentic tool-call path emits surrounding text + reasoning before tool_calls and swaps to the dedicated deepseekWebTools.ts parser; the +8 lines are cohesive wiring at the existing transformSSE chokepoint (the parser itself lives in the new deepseekWebTools.ts file, already under cap). The PR's own fast-gate (PR->release) does not run check:file-size, so this surfaced only at release reconcile. Covered by tests/unit/deepseek-web-tools-variants.test.ts + deepseek-web-tools-execute.test.ts.",
"_rebaseline_2026_06_23_4712_deepseek_web_tool_results": "PR for #4712 (deepseek-web drops role:tool): open-sse/executors/deepseek-web.ts 1125->1148 (+23). messagesToPrompt() now folds role:\"tool\" results into the single-prompt transcript (recovering the tool name from the preceding assistant tool_calls by tool_call_id) instead of silently dropping them; the lines are cohesive wiring inside the existing function. Covered by tests/unit/deepseek-web-tool-result-prompt-4712.test.ts.",

163
open-sse/handlers/chatCore.ts
View File

@@ -165,26 +165,9 @@ import {
getStripTypesForProviderModel,
stripIncompatibleMessageContent,
} from "../services/modelStrip.ts";
import { normalizeMimoThinking } from "../services/mimoThinking.ts";
import {
isOpencodeGoProvider,
stripBooleanReasoning,
} from "../services/opencodeReasoningSanitizer.ts";
import {
normalizeClaudeAdaptiveThinking,
normalizeClaudeDisabledThinkingEffort,
} from "../services/claudeAdaptiveThinking.ts";
import { shouldUseMidConversationSystem } from "../executors/claudeIdentity.ts";
import { normalizeClaudeHaikuConstraints } from "../services/claudeHaikuConstraints.ts";
import { applyDefaultReasoningEffort } from "../services/defaultReasoningEffort.ts";
import { wireAdaptiveEffort } from "./chatCore/adaptiveEffortWiring.ts";
import { echoModelInObject } from "../services/responseModelEcho.ts";
import {
stripGpt5SamplingWhenReasoning,
stripGpt5ReasoningWhenTools,
} from "../services/gpt5SamplingGuard.ts";
import { getUnsupportedParams, REGISTRY } from "../config/providerRegistry.ts";
import { stripUnsupportedParams } from "./chatCore/unsupportedParamsStrip.ts";
import { checkToolCallingRequiredButUnsupported } from "./chatCore/toolCallingRequiredCheck.ts";
import {
supportsMaxTokens,
@@ -210,7 +193,6 @@ import {
isTinyBudgetReasoningProbe,
toPositiveInteger,
} from "../services/reasoningTokenBuffer.ts";
import { normalizeThinkingForModel } from "@/shared/constants/modelSpecs.ts";
import {
buildErrorBody,
createErrorResult,
@@ -512,6 +494,19 @@ export async function handleChatCore({
videoBridgeLog = undefined,
fallbackAttempts = undefined,
}) {
const {
model: originModel,
resolvedThinkingEffort,
defaultThinkingEffort,
} = modelInfo as typeof modelInfo & {
resolvedThinkingEffort?: string | null;
defaultThinkingEffort?: string | null;
};
const trustedEffortContext = Object.freeze({
originModel,
resolvedThinkingEffort,
defaultThinkingEffort,
});
let { provider, model, extendedContext } = modelInfo;
// Keep the selected rule across format conversion, retries and refreshed credentials.
// Each combo leg gets its own execution context; nothing is written to shared accounts.
@@ -2734,77 +2729,6 @@ export async function handleChatCore({
}
translatedBody.model = finalModelToUpstream;
// #3554: a combo/route may substitute the upstream model AFTER the client chose its
// `thinking` value. Claude Code sends `thinking:{type:"disabled"}` for internal calls,
// which claude-fable-5 (adaptive-only) rejects with a 400. Drop the now-invalid value
// when the resolved target model rejects it; models that accept `disabled` are untouched.
if (typeof finalModelToUpstream === "string") {
translatedBody = normalizeThinkingForModel(translatedBody, finalModelToUpstream);
// Claude Opus 4.7+/Fable 5 removed manual extended thinking: `thinking.type:"enabled"`
// or any `thinking.budget_tokens` is a hard 400. Collapse any manual thinking that
// reached this point (passthrough legacy shape, reasoning_effort buckets, per-model
// defaults) to `{type:"adaptive"}` — effort stays on `output_config.effort`. Keyed on
// the resolved upstream model, so it covers every routing mode. See claudeAdaptiveThinking.ts.
translatedBody = normalizeClaudeAdaptiveThinking(translatedBody, finalModelToUpstream);
// Opus 5 allows disabled thinking only through high effort on Anthropic's direct
// Messages API. The helper scopes this constraint to `anthropic` and `claude`;
// GitHub Copilot and Claude Web use separate upstream contracts.
translatedBody = normalizeClaudeDisabledThinkingEffort(
translatedBody,
finalModelToUpstream,
provider
);
// Claude Haiku rejects `thinking.type:"adaptive"` and `output_config.effort`
// (both Sonnet 4.6 / Opus 4.5+ only). Several paths can still emit those
// shapes on a Haiku target — native passthrough, reasoning_effort buckets,
// per-model defaults — so collapse them to a Haiku-valid shape here, after
// model substitution. Mirrors upstream 9router 401d93bd5. See
// services/claudeHaikuConstraints.ts.
translatedBody = normalizeClaudeHaikuConstraints(translatedBody, finalModelToUpstream);
// #6879: per-model default reasoning_effort, injected only when the request
// carries no reasoning field of any shape — an explicit client/combo-leg value
// always wins. Scoped to the OpenAI Chat Completions dispatch shape (the shape
// `reasoning_effort` is native to); unset ModelSpec.defaultReasoningEffort is a
// no-op. #7694: `modelInfo.resolvedThinkingEffort` — set when the request's model
// id carried a `<prefix>/<model>-{effort}` synced-model alias suffix
// (`src/sse/services/model.ts`) — takes priority over the static per-model default.
// The synced catalog's vendor-declared `defaultThinkingEffort` (OpenRouter
// `reasoning.default_effort`, captured by `detectDefaultThinkingEffort`) is the
// lowest-priority default: it only fires when neither the suffix alias nor a
// static operator default exists. See open-sse/services/defaultReasoningEffort.ts.
if (targetFormat === FORMATS.OPENAI) {
translatedBody = applyDefaultReasoningEffort(
translatedBody,
finalModelToUpstream,
(modelInfo as { resolvedThinkingEffort?: string })?.resolvedThinkingEffort,
(modelInfo as { defaultThinkingEffort?: string })?.defaultThinkingEffort
);
}
translatedBody = wireAdaptiveEffort(translatedBody, {
rawBody: body,
clientRawRequest,
targetFormat,
});
}
// Xiaomi MiMo controls reasoning ONLY via `thinking:{type:"enabled"|"disabled"}` and
// rejects unknown/extra params with a strict "400 Param Incorrect". Map OmniRoute's
// OpenAI reasoning signals onto that native shape: reduce any thinking object to
// `{type}` and drop `reasoning_effort`/`reasoning`. See services/mimoThinking.ts.
if (provider === "xiaomi-mimo") {
translatedBody = normalizeMimoThinking(translatedBody);
}
// opencode-go backed providers (ollama-cloud, opencode-go, opencode,
// opencode-zen) use a Go ChatCompletionRequest struct where `reasoning`
// is typed as openai.Reasoning (a structured type). A boolean
// `reasoning: true/false` — valid per the OpenAI API — causes a 400
// "json: cannot unmarshal bool into Go struct field" on the Go side.
// Strip the boolean before forwarding. See opencodeReasoningSanitizer.ts.
if (isOpencodeGoProvider(provider)) {
translatedBody = stripBooleanReasoning(translatedBody);
}
const previousResponseIdPolicy = applyResponsesPreviousResponseIdPolicy(translatedBody, {
mode: settings.responsesPreviousResponseIdMode,
provider,
@@ -2862,43 +2786,6 @@ export async function handleChatCore({
return createErrorResult(400, toolCallingCheck.message!, null, "tool_calling_not_supported");
}
if (unsupported.length > 0) {
const { strippedParams } = stripUnsupportedParams(translatedBody, unsupported);
if (strippedParams.length > 0) {
log?.warn?.(
"PARAMS",
`Stripped unsupported params for ${model}: ${strippedParams.join(", ")}`
);
}
}
// GPT-5 reasoning models (openai Chat Completions) reject temperature/top_p with a 400
// whenever a reasoning effort is active, yet accept them under reasoning_effort=none (the
// GPT-5.1+ default). A static unsupportedParams list can't express that, so strip sampling
// conditionally here. The codex Responses path is already covered by the executor allowlist.
translatedBody = stripGpt5SamplingWhenReasoning(
translatedBody,
provider,
finalModelToUpstream,
log
);
// GPT-5.x reasoning models on the raw openai Chat Completions surface reject function
// `tools` combined with an active `reasoning_effort`: HTTP 400 "Function tools with
// reasoning_effort are not supported ... Please use /v1/responses instead." This used to
// be true for every GPT-5.x model on the plain `openai` provider, but #7242 (targetFormat
// "openai-responses" on GPT_5_6_API_CAPABILITIES) now routes the GPT-5.6 family to
// /v1/responses instead, which accepts tools + reasoning natively — so the strip must not
// fire there. Pass the already-resolved `targetFormat` so the guard gates on the actual
// upstream surface for this request instead of a model-name list. Port of 9router#2540.
translatedBody = stripGpt5ReasoningWhenTools(
translatedBody,
provider,
finalModelToUpstream,
targetFormat,
log
);
// Rename max_tokens to max_completion_tokens if not supported (#1961)
if (!supportsMaxTokens({ provider, model })) {
if (translatedBody.max_tokens !== undefined) {
@@ -3111,7 +2998,9 @@ export async function handleChatCore({
// Namespaced by the calling API key: dedup hands the SAME response object to
// every joiner, so a shared hash across keys is a cross-principal response
// leak (GHSA-6c7w-56xp-wpc6).
const dedupHash = dedupEnabled ? computeRequestHash(dedupRequestBody, apiKeyInfo?.id) : null;
const dedupHash = dedupEnabled
? computeRequestHash(dedupRequestBody, apiKeyInfo?.id, trustedEffortContext)
: null;
const executeProviderRequest = async (modelToCall = effectiveModel, allowDedup = false) => {
const execute = async () => {
@@ -3121,12 +3010,15 @@ export async function handleChatCore({
let bodyToSend = await prepareUpstreamBody({
translatedBody,
modelToCall,
...trustedEffortContext,
provider,
targetFormat,
credentials,
credentials: getExecutionCredentials(),
log,
bypassDefaultToolLimit: isOpencodeClient,
isOpencodeClient,
rawBody: body,
clientRawRequest,
});
// Global System Prompt — SINGLE injection point (post-translation) for
@@ -4524,12 +4416,25 @@ export async function handleChatCore({
// stay aligned if this block ever runs after a path that mutates body.model (e.g. fallback).
try {
const retryModelId = String(translatedBody.model || effectiveModel);
const retryBody = await prepareUpstreamBody({
translatedBody,
modelToCall: retryModelId,
...trustedEffortContext,
provider,
targetFormat,
credentials: getExecutionCredentials(),
log,
bypassDefaultToolLimit: isOpencodeClient,
isOpencodeClient,
rawBody: body,
clientRawRequest,
});
assertManagedLeaseFence(getExecutionConnectionId(getExecutionCredentials()));
const retryResult = normalizeExecutorResult(
await runWithCapture(providerRequestCapture, () =>
executor.execute({
model: retryModelId,
body: translatedBody,
body: retryBody,
stream: upstreamStream,
credentials: getExecutionCredentials(),
signal: streamController.signal,

103
open-sse/handlers/chatCore/upstreamBody.ts
View File

@@ -1,13 +1,7 @@
/**
* chatCore upstream body preparation (Quality Gate v2 / Fase 9 — chatCore god-file decomposition,
* #3501 — first internal sub-slice of executeProviderRequest).
*
* Extracted from handleChatCore's execute() closure: prepares the body actually sent upstream for a
* given target model. Pins the model id, applies the configured payload rules, truncates the tool
* list to the provider's effective limit and injects an OpenAI `prompt_cache_key` for
* caching-capable providers. Pure with respect to handler
* state (returns a fresh body, only logs as a side effect); behaviour is byte-identical to the
* previous inline block. Split into small private steps so each stays under the complexity cap.
* Prepare an isolated body for one upstream attempt. Model constraints and defaults
* run before operator payload rules; target sanitation and tool/cache/image handling
* run afterward. The mutable recovery transcript never receives derived defaults.
*/
import {
@@ -22,8 +16,30 @@ import {
} from "../../utils/cacheControlPolicy.ts";
import { FORMATS } from "../../translator/formats.ts";
import { sanitizeRequestForResolvedTarget } from "../../services/targetRequestSanitizer.ts";
import { normalizeThinkingForModel } from "@/shared/constants/modelSpecs.ts";
import {
normalizeClaudeAdaptiveThinking,
normalizeClaudeDisabledThinkingEffort,
} from "../../services/claudeAdaptiveThinking.ts";
import { normalizeClaudeHaikuConstraints } from "../../services/claudeHaikuConstraints.ts";
import { applyDefaultReasoningEffort } from "../../services/defaultReasoningEffort.ts";
import { normalizeMimoThinking } from "../../services/mimoThinking.ts";
import {
isOpencodeGoProvider,
stripBooleanReasoning,
} from "../../services/opencodeReasoningSanitizer.ts";
import { getUnsupportedParams } from "../../config/providerRegistry.ts";
import { stripUnsupportedParams } from "./unsupportedParamsStrip.ts";
import {
stripGpt5SamplingWhenReasoning,
stripGpt5ReasoningWhenTools,
} from "../../services/gpt5SamplingGuard.ts";
import { wireAdaptiveEffort } from "./adaptiveEffortWiring.ts";
type LoggerLike = { debug?: (...args: unknown[]) => void } | null | undefined;
type LoggerLike =
| { debug?: (...args: unknown[]) => void; warn?: (tag: string, message: string) => void }
| null
| undefined;
type Body = Record<string, unknown>;
type CredentialsLike =
| {
@@ -180,18 +196,74 @@ async function injectPromptCacheKey(
return bodyToSend;
}
export async function prepareUpstreamBody(opts: {
type PrepareUpstreamBodyOptions = {
translatedBody: Body;
modelToCall: string;
provider: string | null | undefined;
targetFormat: string;
credentials: CredentialsLike;
originModel?: string | null;
resolvedThinkingEffort?: string | null;
defaultThinkingEffort?: string | null;
bypassDefaultToolLimit?: boolean;
isOpencodeClient?: boolean;
/** Raw (pre-translation) request body — turn-scoped signals for adaptive effort (#13448). */
rawBody?: { messages?: unknown } | undefined;
/** Incoming client request — read for the x-omniroute-effort header (#13448). */
clientRawRequest?: { headers?: unknown } | undefined;
log?: LoggerLike;
}): Promise<Body> {
};
function normalizeAttemptBody(opts: PrepareUpstreamBodyOptions): Body {
const { translatedBody, modelToCall, provider, targetFormat, log } = opts;
// Capture intent before constraints remove unsupported fields. Removed explicit
// choices must not turn into permission to inject automatic defaults.
const hadExplicitReasoning =
translatedBody.reasoning_effort !== undefined ||
translatedBody.reasoning !== undefined ||
translatedBody.thinking !== undefined;
let bodyToSend: Body = { ...structuredClone(translatedBody), model: modelToCall };
bodyToSend = normalizeThinkingForModel(bodyToSend, modelToCall);
bodyToSend = normalizeClaudeAdaptiveThinking(bodyToSend, modelToCall);
bodyToSend = normalizeClaudeDisabledThinkingEffort(bodyToSend, modelToCall, provider);
bodyToSend = normalizeClaudeHaikuConstraints(bodyToSend, modelToCall);
if (targetFormat === FORMATS.OPENAI && !hadExplicitReasoning) {
const isOriginModel = modelToCall === opts.originModel;
bodyToSend = applyDefaultReasoningEffort(
bodyToSend,
modelToCall,
isOriginModel ? opts.resolvedThinkingEffort : undefined,
isOriginModel ? opts.defaultThinkingEffort : undefined
);
}
// #13448: resolve an "auto" effort (X-OmniRoute-Effort header or ModelSpec default) to a
// concrete level. Runs per attempt, right after applyDefaultReasoningEffort — the same
// position it held inline in chatCore.ts before this chain moved here (#13720); it
// self-scopes to FORMATS.OPENAI and no-ops when the body carries explicit reasoning.
bodyToSend = wireAdaptiveEffort(bodyToSend, {
rawBody: opts.rawBody as Parameters<typeof wireAdaptiveEffort>[1]["rawBody"],
clientRawRequest: opts.clientRawRequest,
targetFormat,
});
if (provider === "xiaomi-mimo") bodyToSend = normalizeMimoThinking(bodyToSend);
if (isOpencodeGoProvider(provider)) bodyToSend = stripBooleanReasoning(bodyToSend);
const { strippedParams } = stripUnsupportedParams(
bodyToSend,
getUnsupportedParams(provider, modelToCall)
);
if (strippedParams.length > 0) {
log?.warn?.(
"PARAMS",
`Stripped unsupported params for ${modelToCall}: ${strippedParams.join(", ")}`
);
}
bodyToSend = stripGpt5SamplingWhenReasoning(bodyToSend, provider, modelToCall, log);
bodyToSend = stripGpt5ReasoningWhenTools(bodyToSend, provider, modelToCall, targetFormat, log);
return bodyToSend;
}
export async function prepareUpstreamBody(opts: PrepareUpstreamBodyOptions): Promise<Body> {
const {
translatedBody,
modelToCall,
provider,
targetFormat,
@@ -201,10 +273,7 @@ export async function prepareUpstreamBody(opts: {
log,
} = opts;
let bodyToSend: Body =
translatedBody.model === modelToCall
? translatedBody
: { ...translatedBody, model: modelToCall };
let bodyToSend = normalizeAttemptBody(opts);
const payloadRuleModel =
typeof bodyToSend.model === "string" && bodyToSend.model.length > 0
? bodyToSend.model

28
open-sse/services/requestDedup.ts
View File

@@ -156,7 +156,15 @@ function extractSystemContent(body: Record<string, unknown>): unknown {
* deployments have no tenant boundary to preserve, and every such install would
* otherwise silently lose dedup.
*/
export function computeRequestHash(requestBody: unknown, tenantId?: string | null): string {
export function computeRequestHash(
requestBody: unknown,
tenantId?: string | null,
trustedContext?: {
originModel?: string | null;
resolvedThinkingEffort?: string | null;
defaultThinkingEffort?: string | null;
}
): string {
const body = requestBody as Record<string, unknown>;
const canonical = {
model: body.model ?? null,
@@ -170,6 +178,24 @@ export function computeRequestHash(requestBody: unknown, tenantId?: string | nul
top_p: body.top_p ?? null,
frequency_penalty: body.frequency_penalty ?? null,
presence_penalty: body.presence_penalty ?? null,
...(trustedContext === undefined
? {}
: {
// Read the translated request before attempt constraints can erase intent.
// JSON omission preserves absent/undefined versus null/false; nested key
// order follows the same serialization contract as the legacy projection.
requestIntent: {
reasoning_effort: body.reasoning_effort,
reasoning: body.reasoning,
thinking: body.thinking,
output_config: body.output_config,
},
trustedContext: {
originModel: trustedContext.originModel ?? null,
resolvedThinkingEffort: trustedContext.resolvedThinkingEffort ?? null,
defaultThinkingEffort: trustedContext.defaultThinkingEffort ?? null,
},
}),
};
const digest = createHash("sha256").update(JSON.stringify(canonical)).digest("hex").slice(0, 16);
return tenantId ? `${tenantId}.${digest}` : digest;

24
src/sse/handlers/chat.ts
View File

@@ -1528,24 +1528,13 @@ async function handleSingleModelChat(
customModelTargetFormat,
extendedContext,
apiFormat,
resolvedThinkingEffort,
} = resolved;
// Prefer the combo target's providerId when available — the model string's
// provider prefix may differ from the credential provider ID (e.g. model
// "xiaomi/mimo-v2-flash" resolves to provider "xiaomi" but the combo target
// may specify providerId: "opengate" for credential lookup).
// Guard: if runtimeOptions.providerId is merely the prefix already encoded in
// the model string (e.g. "p2" from "p2/test-model"), and resolveModelOrError
// expanded it to a full custom-node ID (e.g. "openai-compatible-chat-e2e-p2"),
// trust resolvedProvider so the executor receives the full node ID and can
// correctly resolve the custom baseUrl. (#3058 follow-up)
// Use explicit credential redirects, but preserve resolved node IDs for implicit prefixes.
const provider = (() => {
if (!runtimeOptions.providerId) return resolvedProvider;
// If the override is identical to resolvedProvider, no-op.
if (runtimeOptions.providerId === resolvedProvider) return resolvedProvider;
// If the model string already encodes runtimeOptions.providerId as its prefix,
// the override is implicit (not an intentional redirect) — use resolvedProvider.
if (modelStr.startsWith(runtimeOptions.providerId + "/")) return resolvedProvider;
// Intentional override (e.g. providerId points to a different credential pool).
return runtimeOptions.providerId;
})();
const forceLiveComboTest = runtimeOptions.forceLiveComboTest === true;
@@ -1986,10 +1975,11 @@ async function handleSingleModelChat(
runtimeOptions.comboExecutionKey ?? runtimeOptions.comboStepId ?? null,
extendedContext,
modelApiFormat: apiFormat,
// Only a model's explicit DB override may cross this boundary as
// modelInfo.targetFormat. The effective targetFormat above was
// resolved without credentials; forwarding it would let a stale
// provider-id fallback override the credential-aware resolution.
resolvedThinkingEffort:
effectiveModel === model && provider === resolvedProvider
? resolvedThinkingEffort
: undefined,
// Forward only the DB override, not the credential-blind format fallback.
modelTargetFormat: customModelTargetFormat,
providerProfile,
cachedSettings: runtimeOptions.cachedSettings,

3
src/sse/handlers/chatHelpers.ts
View File

@@ -350,6 +350,7 @@ export async function resolveModelOrError(
customModelTargetFormat,
extendedContext,
apiFormat,
resolvedThinkingEffort: modelInfo.resolvedThinkingEffort,
};
}
@@ -444,6 +445,7 @@ export async function executeChatWithBreaker({
extendedContext,
modelApiFormat,
modelTargetFormat,
resolvedThinkingEffort,
providerProfile,
cachedSettings,
skipUpstreamRetry = false,
@@ -497,6 +499,7 @@ export async function executeChatWithBreaker({
extendedContext,
apiFormat: modelApiFormat,
targetFormat: modelTargetFormat,
resolvedThinkingEffort,
},
credentials: refreshedCredentials,
log: handlerLog,

376
tests/unit/chatcore-upstream-body.test.ts
View File

@@ -19,6 +19,10 @@ const { FORMATS } = await import("../../open-sse/translator/formats.ts");
const { setParamFilterConfig, deleteParamFilterConfig } =
await import("../../src/lib/db/paramFilters.ts");
const { MODEL_SPECS } = await import("../../src/shared/constants/modelSpecs.ts");
const { setPayloadRulesConfig, resetPayloadRulesConfigForTests } =
await import("../../open-sse/services/payloadRules.ts");
before(async () => {
await coreDb.ensureDbInitialized();
});
@@ -28,6 +32,378 @@ after(() => {
fs.rmSync(testDataDir, { recursive: true, force: true, maxRetries: 5, retryDelay: 100 });
});
test("explicit hash intent follows translated reasoning representations before preparation", async () => {
const { computeRequestHash } = await import("../../open-sse/services/requestDedup.ts");
const variants = [
{
source: FORMATS.CLAUDE,
target: FORMATS.OPENAI,
model: "gpt-5.2",
provider: "openai",
field: "reasoning_effort",
},
{
source: FORMATS.OPENAI,
target: FORMATS.CLAUDE,
model: "claude-opus-4.7",
provider: "anthropic",
field: "output_config",
},
{
source: FORMATS.OPENAI,
target: FORMATS.OPENAI_RESPONSES,
model: "gpt-5.2",
provider: "openai",
field: "reasoning",
},
];
for (const variant of variants) {
const translated = ["low", "high"].map(
(effort) =>
translateRequest(
variant.source,
variant.target,
variant.model,
{
model: variant.model,
messages: [{ role: "user", content: "Compare translated intent." }],
...(variant.source === FORMATS.CLAUDE
? { output_config: { effort } }
: { reasoning_effort: effort }),
},
false,
null,
variant.provider
) as Record<string, unknown>
);
assert.ok(
translated.every((request) => request[variant.field] !== undefined),
variant.field
);
// Hold unrelated translated caps/sampling constant to isolate the intent projection.
const requests = translated.map((request) => ({
model: variant.model,
messages: bodyMessages(),
reasoning_effort: request.reasoning_effort,
reasoning: request.reasoning,
thinking: request.thinking,
output_config: request.output_config,
}));
assert.notEqual(
computeRequestHash(requests[0], "tenant", {}),
computeRequestHash(requests[1], "tenant", {}),
variant.field
);
}
function bodyMessages() {
return [{ role: "user", content: "Compare translated intent." }];
}
});
test("automatic effort is limited to OpenAI chat bodies", async () => {
for (const targetFormat of ["claude", "openai-responses", "gemini"]) {
const out = await prepareUpstreamBody({
translatedBody: {},
modelToCall: "attempt-format-fixture",
provider: "test",
targetFormat,
credentials: null,
originModel: "attempt-format-fixture",
resolvedThinkingEffort: "high",
defaultThinkingEffort: "low",
});
assert.equal(out.reasoning_effort, undefined);
}
});
test("attempt constraints rewrite Claude fields without changing the source", async () => {
const source = {
thinking: { type: "enabled", budget_tokens: 10000 },
output_config: { effort: "max", format: "text" },
};
const before = structuredClone(source);
const options = {
translatedBody: source,
provider: "anthropic",
targetFormat: "claude",
credentials: null,
originModel: "claude-opus-4.7",
resolvedThinkingEffort: "high",
};
const adaptive = await prepareUpstreamBody({ ...options, modelToCall: "claude-opus-4.7" });
assert.deepEqual(adaptive.thinking, { type: "adaptive" });
assert.equal(adaptive.reasoning_effort, undefined);
const haiku = await prepareUpstreamBody({
...options,
modelToCall: "claude-haiku-4.5",
translatedBody: { ...source, thinking: { type: "adaptive" } },
});
assert.deepEqual(haiku.thinking, { type: "enabled", budget_tokens: 10000 });
assert.deepEqual(haiku.output_config, { format: "text" });
const disabled = await prepareUpstreamBody({
...options,
modelToCall: "claude-opus-5",
translatedBody: { ...source, thinking: { type: "disabled" } },
});
assert.deepEqual(disabled.output_config, { effort: "high", format: "text" });
assert.deepEqual(source, before);
});
test("attempt effort leaves sampling and recovered history reusable for replacement models", async () => {
const source = {
model: "gpt-5.2",
temperature: 0.3,
top_p: 0.8,
messages: [{ role: "user", content: "recovered history" }],
};
const before = structuredClone(source);
const options = {
translatedBody: source,
provider: "openai",
targetFormat: "openai",
credentials: null,
originModel: "gpt-5.2",
resolvedThinkingEffort: "high",
defaultThinkingEffort: "low",
};
const first = await prepareUpstreamBody({ ...options, modelToCall: "gpt-5.2" });
assert.equal(first.reasoning_effort, "high");
assert.equal(first.temperature, undefined);
assert.equal(first.top_p, undefined);
const replacement = await prepareUpstreamBody({ ...options, modelToCall: "gpt-5.1" });
assert.equal(replacement.reasoning_effort, undefined);
assert.equal(replacement.temperature, 0.3);
assert.equal(replacement.top_p, 0.8);
assert.deepEqual(replacement.messages, source.messages);
assert.deepEqual(source, before);
assert.notEqual(first, source);
});
test("static defaults belong to the attempt; suffix and synced defaults belong to the origin", async () => {
const model = "gpt-5-attempt-fixture";
MODEL_SPECS[model] = { defaultReasoningEffort: "low" };
try {
const options = {
translatedBody: { temperature: 0.3 },
modelToCall: model,
provider: "openai",
targetFormat: "openai",
credentials: null,
originModel: "original",
resolvedThinkingEffort: "high",
defaultThinkingEffort: "max",
};
const replacement = await prepareUpstreamBody(options);
assert.equal(replacement.reasoning_effort, "low");
assert.equal(replacement.temperature, undefined);
const original = await prepareUpstreamBody({ ...options, originModel: model });
assert.equal(original.reasoning_effort, "high");
const staticOnly = await prepareUpstreamBody({
...options,
originModel: model,
resolvedThinkingEffort: null,
});
assert.equal(staticOnly.reasoning_effort, "low");
delete MODEL_SPECS[model];
const syncedOnly = await prepareUpstreamBody({
...options,
originModel: model,
resolvedThinkingEffort: null,
});
assert.equal(syncedOnly.reasoning_effort, "max");
} finally {
delete MODEL_SPECS[model];
}
});
for (const choice of [
{ thinking: { type: "disabled" } },
{ thinking: false },
{ thinking: null },
{ thinking: {} },
{ reasoning: false },
{ reasoning: null },
{ reasoning: {} },
{ reasoning_effort: "none" },
{ reasoning_effort: null },
]) {
test(`explicit intent precedes destructive constraints: ${JSON.stringify(choice)}`, async () => {
const model = "claude-fable-5";
const prior = MODEL_SPECS[model];
MODEL_SPECS[model] = { ...prior, defaultReasoningEffort: "medium" };
try {
const source = structuredClone(choice);
const out = await prepareUpstreamBody({
translatedBody: source,
modelToCall: model,
provider: "cheaperinference",
targetFormat: "openai",
credentials: null,
originModel: model,
resolvedThinkingEffort: "high",
defaultThinkingEffort: "low",
});
assert.equal(
out.reasoning_effort,
"reasoning_effort" in choice ? choice.reasoning_effort : undefined
);
if (
"thinking" in choice &&
choice.thinking &&
typeof choice.thinking === "object" &&
"type" in choice.thinking
)
assert.equal(out.thinking, undefined);
assert.deepEqual(source, choice);
} finally {
MODEL_SPECS[model] = prior;
}
});
}
for (const metadata of [
{},
{ originModel: null, resolvedThinkingEffort: null, defaultThinkingEffort: null },
]) {
test(`empty bodies and absent metadata stay compatible: ${JSON.stringify(metadata)}`, async () => {
const source = {};
const out = await prepareUpstreamBody({
translatedBody: source,
modelToCall: "unconfigured-model",
provider: null,
targetFormat: "openai",
credentials: undefined,
...metadata,
});
assert.deepEqual(out, { model: "unconfigured-model" });
assert.deepEqual(source, {});
assert.notEqual(out, source);
});
}
test("attempt constraints isolate nested tool history and recompute registry restrictions", async () => {
const source = {
model: "o3",
temperature: 0.2,
tools: [{ type: "function", function: { name: "lookup" } }],
messages: [
{
role: "assistant",
content: null,
tool_calls: [
{ id: "call1", type: "function", function: { name: "lookup", arguments: "{}" } },
],
},
{ role: "tool", tool_call_id: "call1", content: "result" },
],
};
const before = structuredClone(source);
const options = { translatedBody: source, targetFormat: "openai", credentials: null };
const restricted = await prepareUpstreamBody({
...options,
provider: "aihorde",
modelToCall: "worker",
});
assert.equal(restricted.tools, undefined);
assert.ok((restricted.messages as Array<{ role: string }>).every((m) => m.role !== "tool"));
const open = await prepareUpstreamBody({ ...options, provider: "openai", modelToCall: "gpt-4o" });
assert.deepEqual(open.tools, source.tools);
assert.deepEqual(open.messages, source.messages);
assert.equal(open.temperature, 0.2);
const reasoning = await prepareUpstreamBody({
...options,
provider: "openai",
modelToCall: "o3",
});
assert.equal(reasoning.temperature, undefined);
assert.deepEqual(source, before);
});
test("payload rules run after automatic guards and before target sanitation", async () => {
setPayloadRulesConfig({
default: [{ models: [{ name: "*" }], params: { reasoning_effort: "low" } }],
override: [
{
models: [{ name: "*" }],
params: { reasoning_effort: "high", temperature: 0.7, verbosity: "low" },
},
],
filter: [{ models: [{ name: "*" }], params: ["top_p"] }],
});
try {
const source = {
temperature: 0.2,
top_p: 0.9,
tools: [{ type: "function", function: { name: "lookup" } }],
};
const opts = {
translatedBody: source,
provider: "openai",
targetFormat: "openai",
credentials: null,
originModel: "gpt-5.2",
resolvedThinkingEffort: "high",
};
const origin = await prepareUpstreamBody({ ...opts, modelToCall: "gpt-5.2" });
assert.equal(origin.reasoning_effort, "high");
assert.equal(origin.temperature, 0.7);
assert.equal(origin.top_p, undefined);
const replacement = await prepareUpstreamBody({
...opts,
modelToCall: "other-model",
provider: "opencode-go",
});
assert.equal(replacement.reasoning_effort, "high");
assert.equal(replacement.verbosity, undefined);
assert.equal(source.temperature, 0.2);
} finally {
resetPayloadRulesConfigForTests();
}
});
test("sampling guard precedes the function tools guard", async () => {
const source = {
temperature: 0.4,
top_p: 0.8,
tools: [{ type: "function", function: { name: "lookup" } }],
};
const out = await prepareUpstreamBody({
translatedBody: source,
modelToCall: "gpt-5.2",
provider: "openai",
targetFormat: "openai",
credentials: null,
originModel: "gpt-5.2",
resolvedThinkingEffort: "high",
});
assert.equal(out.reasoning_effort, undefined);
assert.equal(out.temperature, undefined);
assert.equal(out.top_p, undefined);
assert.equal(source.temperature, 0.4);
});
for (const provider of ["xiaomi-mimo", "opencode-go"]) {
test(`${provider} cleanup runs after default selection without restoring explicit choices`, async () => {
const options = {
modelToCall: "unknown-model",
originModel: "unknown-model",
provider,
targetFormat: "openai",
credentials: null,
resolvedThinkingEffort: "high",
};
const out = await prepareUpstreamBody({ ...options, translatedBody: { reasoning: false } });
assert.equal(out.reasoning, undefined);
assert.equal(out.reasoning_effort, undefined);
if (provider === "xiaomi-mimo") {
const bare = await prepareUpstreamBody({ ...options, translatedBody: {} });
assert.equal(bare.reasoning_effort, undefined);
assert.equal(bare.thinking, undefined);
}
});
}
test("pins the target model when it differs from the translated body model", async () => {
const out = await prepareUpstreamBody({
translatedBody: { model: "model-a", messages: [] },

143
tests/unit/request-dedup-tenant-isolation.test.ts
View File

@@ -24,6 +24,149 @@ const body = {
stream: false,
};
test("trusted effort contexts split plain, low and high requests", () => {
const hashes = [null, "low", "high"].map((resolvedThinkingEffort) =>
computeRequestHash(body, "tenant", {
originModel: "original",
resolvedThinkingEffort,
})
);
assert.equal(new Set(hashes).size, 3);
});
test("trusted hash context has fixed keys and null defaults", () => {
assert.equal(
computeRequestHash(body, "tenant", {}),
computeRequestHash(body, "tenant", {
defaultThinkingEffort: null,
resolvedThinkingEffort: null,
originModel: null,
})
);
assert.equal(
computeRequestHash(body, "tenant", { originModel: "original", resolvedThinkingEffort: "high" }),
computeRequestHash(body, "tenant", {
resolvedThinkingEffort: "high",
originModel: "original",
defaultThinkingEffort: undefined,
})
);
for (const key of ["originModel", "resolvedThinkingEffort", "defaultThinkingEffort"] as const) {
assert.notEqual(
computeRequestHash(body, "tenant", {}),
computeRequestHash(body, "tenant", { [key]: "different" })
);
}
assert.notEqual(computeRequestHash(body, "tenant", {}), computeRequestHash(body, "other", {}));
assert.notEqual(computeRequestHash(body, "tenant"), computeRequestHash(body, "tenant", {}));
});
test("legacy digest projection is unchanged without trusted context", async () => {
const { createHash } = await import("node:crypto");
const expected = createHash("sha256")
.update(
JSON.stringify({
model: body.model,
messages: body.messages,
system: null,
temperature: 0,
tools: null,
tool_choice: null,
max_tokens: null,
response_format: null,
top_p: null,
frequency_penalty: null,
presence_penalty: null,
})
)
.digest("hex")
.slice(0, 16);
assert.equal(computeRequestHash(body), expected);
assert.equal(computeRequestHash(body, "tenant"), `tenant.${expected}`);
assert.equal(
computeRequestHash({ ...body, trustedContext: { resolvedThinkingEffort: "high" } }, "tenant"),
`tenant.${expected}`
);
});
for (const field of ["reasoning_effort", "reasoning", "thinking", "output_config"]) {
test(`explicit hash intent preserves ${field} presence and complete values`, () => {
const values = [undefined, null, false, {}, "none", "high", { effort: "high" }];
const hashes = values.map((value) =>
computeRequestHash({ ...body, [field]: value }, "tenant", {})
);
assert.equal(new Set(hashes).size, values.length);
assert.equal(hashes[0], computeRequestHash(body, "tenant", {}));
for (const value of values) {
const request = { ...body, [field]: value };
assert.equal(
computeRequestHash(request, "tenant", {}),
computeRequestHash(structuredClone(request), "tenant", {})
);
assert.notEqual(
computeRequestHash(request, "tenant", {}),
computeRequestHash(request, "other", {})
);
assert.equal(computeRequestHash(request, "tenant"), computeRequestHash(body, "tenant"));
}
});
}
for (const [field, first, second] of [
["reasoning", { effort: "high", summary: "auto" }, { effort: "high", summary: "detailed" }],
["thinking", { type: "enabled", budget_tokens: 1024 }, { type: "enabled", budget_tokens: 2048 }],
["output_config", { effort: "high" }, { effort: "max" }],
[
"output_config",
{ effort: "high", format: { type: "text" } },
{ effort: "high", format: { type: "json" } },
],
] as const) {
test(`explicit hash intent preserves nested ${field} knobs ${JSON.stringify(second)}`, () => {
assert.notEqual(
computeRequestHash({ ...body, [field]: first }, "tenant", {}),
computeRequestHash({ ...body, [field]: second }, "tenant", {})
);
});
}
test("explicit hash intent uses fixed outer keys and ordinary nested JSON order", () => {
const a = { ...body, reasoning: { effort: "high", summary: "auto" }, thinking: false };
const b = { thinking: false, reasoning: { effort: "high", summary: "auto" }, ...body };
assert.equal(computeRequestHash(a, "tenant", {}), computeRequestHash(b, "tenant", {}));
assert.notEqual(
computeRequestHash(a, "tenant", {}),
computeRequestHash({ ...a, reasoning: { summary: "auto", effort: "high" } }, "tenant", {})
);
assert.notEqual(
computeRequestHash({ ...body, reasoning: { knobs: [1, 2] } }, "tenant", {}),
computeRequestHash({ ...body, reasoning: { knobs: [2, 1] } }, "tenant", {})
);
});
test("explicit hash intent cannot spoof origin metadata and retains suffix separation", () => {
const request = { ...body, reasoning_effort: "high" };
const trusted = { originModel: "original", resolvedThinkingEffort: "low" };
assert.equal(
computeRequestHash(request, "tenant", trusted),
computeRequestHash(
{
...request,
trustedContext: { resolvedThinkingEffort: "high" },
requestIntent: { reasoning_effort: "none" },
originModel: "spoofed",
resolvedThinkingEffort: "high",
},
"tenant",
trusted
)
);
assert.notEqual(
computeRequestHash(request, "tenant", trusted),
computeRequestHash(request, "tenant", { ...trusted, resolvedThinkingEffort: "high" })
);
});
test("the same request from two different API keys does NOT share a dedup hash", () => {
const hashKey1 = computeRequestHash(body, "apikey-1");
const hashKey2 = computeRequestHash(body, "apikey-2");

571
tests/unit/suffix-effort-propagation.test.ts Normal file
View File

@@ -0,0 +1,571 @@
import test from "node:test";
import assert from "node:assert/strict";
import { createServer } from "node:http";
import { once } from "node:events";
const networkFetch = globalThis.fetch;
import { createChatPipelineHarness } from "../integration/_chatPipelineHarness.ts";
const harness = await createChatPipelineHarness("suffix-effort-propagation");
const { handleChat, buildRequest, buildOpenAIResponse, seedConnection, combosDb } = harness;
const { persistDiscoveredModels } = await import("../../src/lib/providerModels/modelDiscovery.ts");
const { resolveModelOrError } = await import("../../src/sse/handlers/chatHelpers.ts");
const { setBackgroundDegradationConfig } =
await import("../../open-sse/services/backgroundTaskDetector.ts");
const { setCustomAliases } = await import("../../open-sse/services/modelDeprecation.ts");
const MODEL = "agnes-3.0-flash";
test.beforeEach(async () => {
await harness.resetStorage();
const connection = await seedConnection("agnes");
await persistDiscoveredModels("agnes", connection.id, [
{ id: MODEL, reasoning: { supported_efforts: ["none", "low", "high", "max"] } },
]);
});
test.after(async () => {
await harness.cleanup();
});
test("resolution retains the validated suffix effort", async () => {
const result = await resolveModelOrError(`agnes/${MODEL}-max`, {
messages: [{ role: "user", content: "hello" }],
});
assert.equal(result.model, MODEL);
assert.equal(result.resolvedThinkingEffort, "max");
});
for (const combo of [false, true]) {
for (const effort of ["none", "max"]) {
test(`${combo ? "combo" : "direct"} ${effort} suffix reaches the upstream body`, async () => {
const target = `agnes/${MODEL}-${effort}`;
const model = combo ? "suffix-effort-combo" : target;
if (combo) {
await combosDb.createCombo({ name: model, strategy: "priority", models: [target] });
}
const bodies: Record<string, unknown>[] = [];
globalThis.fetch = async (_input, init) => {
bodies.push(JSON.parse(String(init?.body)));
return buildOpenAIResponse("The requested calculation is complete.", MODEL);
};
const response = await handleChat(
buildRequest({
body: {
model,
stream: false,
messages: [{ role: "user", content: "Compute a short example." }],
},
})
);
assert.equal(response.status, 200, await response.text());
assert.equal(bodies.length, 1);
assert.equal(bodies[0].model, MODEL);
assert.equal(bodies[0].reasoning_effort, effort);
});
}
}
test("explicit client effort still takes precedence over the suffix", async () => {
let sent: Record<string, unknown> | undefined;
globalThis.fetch = async (_input, init) => {
sent = JSON.parse(String(init?.body));
return buildOpenAIResponse("The requested calculation is complete.", MODEL);
};
const response = await handleChat(
buildRequest({
body: {
model: `agnes/${MODEL}-max`,
stream: false,
reasoning_effort: "low",
messages: [{ role: "user", content: "Compute a short example." }],
},
})
);
assert.equal(response.status, 200, await response.text());
assert.equal(sent?.reasoning_effort, "low");
});
for (const choice of [{ reasoning: false }, { thinking: { type: "disabled" } }, {}]) {
test(`unsuffixed or explicit reasoning stays unchanged: ${JSON.stringify(choice)}`, async () => {
let sent: Record<string, unknown> | undefined;
globalThis.fetch = async (_input, init) => {
sent = JSON.parse(String(init?.body));
return buildOpenAIResponse("The requested calculation is complete.", MODEL);
};
const response = await handleChat(
buildRequest({
body: {
model: `agnes/${MODEL}${Object.keys(choice).length ? "-max" : ""}`,
stream: false,
...choice,
messages: [{ role: "user", content: "Compute a short example." }],
},
})
);
assert.equal(response.status, 200, await response.text());
assert.equal(sent?.reasoning_effort, undefined);
for (const [field, value] of Object.entries(choice)) assert.deepEqual(sent?.[field], value);
});
}
test("connection default replacement does not inherit a bare alias suffix", async () => {
await harness.resetStorage();
const connection = await seedConnection("agnes");
const { updateProviderConnection } = await import("../../src/lib/db/providers.ts");
await updateProviderConnection(String(connection.id), {
defaultModel: "agnes-default-replacement",
});
await persistDiscoveredModels("agnes", String(connection.id), [
{ id: MODEL, reasoning: { supported_efforts: ["high", "max"] } },
]);
await harness.settingsDb.updateSettings({
modelAliases: { "bare-effort-alias": `agnes/${MODEL}-max` },
});
let sent: Record<string, unknown> | undefined;
globalThis.fetch = async (_input, init) => {
sent = JSON.parse(String(init?.body));
return buildOpenAIResponse("The replacement answered.", "agnes-default-replacement");
};
const resolved = await resolveModelOrError("bare-effort-alias", {
messages: [{ role: "user", content: "hello" }],
});
assert.equal(resolved.resolvedThinkingEffort, "max");
const response = await handleChat(
buildRequest({
body: {
model: "bare-effort-alias",
stream: false,
messages: [{ role: "user", content: "Compute a short example." }],
},
})
);
assert.equal(response.status, 200, await response.text());
assert.equal(sent?.model, "agnes-default-replacement");
assert.equal(sent?.reasoning_effort, undefined);
});
test("a different combo provider does not inherit the original suffix", async () => {
await seedConnection("openai");
await combosDb.createCombo({
name: "redirect-effort",
strategy: "priority",
models: [{ id: "redirect", kind: "model", providerId: "openai", model: `agnes/${MODEL}-max` }],
});
let sent: Record<string, unknown> | undefined;
globalThis.fetch = async (_input, init) => {
sent = JSON.parse(String(init?.body));
return buildOpenAIResponse("The requested calculation is complete.", MODEL);
};
const response = await handleChat(
buildRequest({
body: {
model: "redirect-effort",
stream: false,
messages: [{ role: "user", content: "Compute a short example." }],
},
})
);
assert.equal(response.status, 200, await response.text());
assert.equal(sent?.reasoning_effort, undefined);
});
for (const redirect of ["background", "alias"]) {
for (const explicit of [false, true]) {
test(`${redirect} replacement drops suffix but preserves explicit effort: ${explicit}`, async () => {
const replacement = "agnes-review-cheap";
let sent: Record<string, unknown> | undefined;
globalThis.fetch = async (_input, init) => {
sent = JSON.parse(String(init?.body));
return buildOpenAIResponse("The requested calculation is complete.", replacement);
};
try {
if (redirect === "background") {
setBackgroundDegradationConfig({
enabled: true,
degradationMap: { [MODEL]: replacement },
});
} else {
setCustomAliases({ [MODEL]: replacement });
}
const response = await handleChat(
buildRequest({
headers: redirect === "background" ? { "x-task-type": "background" } : {},
body: {
model: `agnes/${MODEL}-max`,
stream: false,
...(explicit ? { reasoning_effort: "low" } : {}),
messages: [{ role: "user", content: "Compute a short example." }],
},
})
);
assert.equal(response.status, 200, await response.text());
assert.equal(sent?.model, replacement);
assert.equal(sent?.reasoning_effort, explicit ? "low" : undefined);
} finally {
setBackgroundDegradationConfig({ enabled: false, degradationMap: {} });
setCustomAliases({});
}
});
}
}
for (const failure of ["empty"]) {
for (const mode of ["plain", "explicit", "suffix"]) {
test(`${failure} family fallback preserves only explicit effort: ${mode}`, async () => {
const provider = "cheaperinference";
const model = "claude-sonnet-4.6";
const fallback = "claude-sonnet-4.5";
const connection = await seedConnection(provider);
await persistDiscoveredModels(provider, connection.id, [
{ id: model, reasoning: { supported_efforts: ["low", "high"] } },
{ id: fallback, reasoning: { supported_efforts: ["low", "high"] } },
]);
const bodies: Record<string, unknown>[] = [];
globalThis.fetch = async (_url, init) => {
const sent = JSON.parse(String(init?.body));
bodies.push(sent);
if (sent.model === model) return buildOpenAIResponse("", model);
return buildOpenAIResponse("The requested example is complete.", sent.model);
};
const response = await handleChat(
buildRequest({
body: {
model: `${provider}/${model}${mode === "suffix" ? "-high" : ""}`,
stream: false,
...(mode === "explicit" ? { reasoning_effort: "low" } : {}),
messages: [{ role: "user", content: "Explain a small example." }],
},
})
);
assert.equal(response.status, 200, await response.text());
assert.equal(bodies[0].model, model);
assert.equal(bodies.at(-1)?.model, fallback);
assert.equal(
bodies[0].reasoning_effort,
mode === "suffix" ? "high" : mode === "explicit" ? "low" : undefined
);
assert.equal(bodies.at(-1)?.reasoning_effort, mode === "explicit" ? "low" : undefined);
});
}
}
test("server tool follow-up materializes suffix effort on the fresh translated transcript", async () => {
const apiKey = await harness.seedApiKey();
await harness.settingsDb.updateSettings({ skillsEnabled: true });
const { encodeSkillToolName } = await import("../../src/lib/skills/injection.ts");
harness.skillExecutor.registerHandler("attempt-weather", async () => ({ forecast: "sunny" }));
await harness.skillRegistry.register({
apiKeyId: apiKey.id,
name: "lookupWeather",
version: "1.0.0",
description: "Weather fixture",
schema: {
input: { type: "object", properties: { location: { type: "string" } } },
output: { type: "object" },
},
handler: "attempt-weather",
enabled: true,
});
const previous = process.env.SERVER_OWNED_TOOL_LOOP_ENABLED;
process.env.SERVER_OWNED_TOOL_LOOP_ENABLED = "true";
const bodies: Record<string, unknown>[] = [];
globalThis.fetch = async (_input, init) => {
const sent = JSON.parse(String(init?.body));
bodies.push(sent);
if (bodies.length === 1)
return harness.buildOpenAIToolCallResponse({
model: MODEL,
toolCallId: "attempt-weather-call",
toolName: encodeSkillToolName("lookupWeather", "1.0.0"),
argumentsObject: { location: "Tokyo" },
});
return buildOpenAIResponse("The weather is sunny.", MODEL);
};
try {
const response = await handleChat(
buildRequest({
authKey: apiKey.key,
body: {
model: `agnes/${MODEL}-high`,
stream: false,
messages: [{ role: "user", content: "Use lookupWeather for Tokyo." }],
},
})
);
assert.equal(response.status, 200);
const payload = await response.json();
assert.equal(payload.choices[0].message.content, "The weather is sunny.");
assert.equal(bodies.length, 2);
assert.equal(bodies[0].reasoning_effort, "high");
assert.equal(bodies[1].reasoning_effort, "high");
const messages = bodies[1].messages as Array<Record<string, unknown>>;
assert.ok(
messages.some(
(message) => message.role === "tool" && message.tool_call_id === "attempt-weather-call"
)
);
} finally {
if (previous === undefined) delete process.env.SERVER_OWNED_TOOL_LOOP_ENABLED;
else process.env.SERVER_OWNED_TOOL_LOOP_ENABLED = previous;
}
});
test("streaming direct refresh retry prepares effort after the pipeline refresh was exhausted", async () => {
const { handleChatCore } = await import("../../open-sse/handlers/chatCore.ts");
const bodies: Record<string, unknown>[] = [];
const headers: Headers[] = [];
let refreshes = 0;
globalThis.fetch = async (input, init) => {
if (String(input).startsWith("https://api.github.com/copilot_internal/v2/token")) {
refreshes++;
modelInfo.model = "mutated-caller-model";
modelInfo.resolvedThinkingEffort = "low";
return Response.json({
token: `fixture-refresh-${refreshes}`,
expires_at: Math.floor(Date.now() / 1000) + 3600,
});
}
bodies.push(JSON.parse(String(init?.body)));
headers.push(new Headers(init?.headers));
if (bodies.length < 3)
return Response.json({ error: { message: "Expired token" } }, { status: 401 });
return new Response(
`data: ${JSON.stringify({ id: "refresh-fixture", object: "chat.completion.chunk", choices: [{ index: 0, delta: { role: "assistant", content: "Refresh succeeded." }, finish_reason: "stop" }] })}\n\ndata: [DONE]\n\n`,
{ headers: { "Content-Type": "text/event-stream" } }
);
};
const modelInfo = { provider: "github", model: "gpt-4o-mini", resolvedThinkingEffort: "high" };
const result = await handleChatCore({
body: {
model: "gpt-4o-mini",
stream: true,
messages: [{ role: "user", content: "Exercise refresh retry." }],
},
modelInfo,
credentials: {
accessToken: "fixture-access",
refreshToken: "fixture-refresh",
providerSpecificData: { copilotToken: "fixture-initial" },
},
log: { debug() {}, info() {}, warn() {}, error() {} },
onCredentialsRefreshed: async () => {},
onRequestSuccess: undefined,
onStreamFailure: undefined,
onDisconnect: undefined,
clientRawRequest: undefined,
connectionId: undefined,
userAgent: undefined,
comboName: undefined,
});
assert.ok(!(result instanceof Response));
assert.equal(result.success, true);
assert.match(await result.response.text(), /Refresh succeeded/);
assert.equal(bodies.length, 3);
assert.equal(refreshes, 2);
for (const body of bodies) assert.equal(body.reasoning_effort, "high");
assert.equal(headers[2].get("authorization"), "Bearer fixture-refresh-2");
});
for (const pair of [
["high", "high"],
["low", "high"],
]) {
test(`overlapping suffix requests ${pair.join("/")} preserve dedup identity over HTTP`, async () => {
const bodies: Record<string, unknown>[] = [];
const server = createServer(async (req, res) => {
let text = "";
for await (const chunk of req) text += chunk;
const sent = JSON.parse(text);
bodies.push(sent);
await new Promise((resolve) => setTimeout(resolve, 350));
res.setHeader("Content-Type", "application/json");
res.end(
await buildOpenAIResponse(`Completed with ${sent.reasoning_effort} effort.`, MODEL).text()
);
});
server.listen(0, "127.0.0.1");
await once(server, "listening");
try {
const address = server.address();
assert.ok(address && typeof address !== "string");
globalThis.fetch = (_input, init) =>
networkFetch(`http://127.0.0.1:${address.port}/v1/chat/completions`, init);
const responses = await Promise.all(
pair.map((effort) =>
handleChat(
buildRequest({
body: {
model: `agnes/${MODEL}-${effort}`,
stream: false,
temperature: 0,
messages: [{ role: "user", content: "Compute an overlapping example." }],
},
})
)
)
);
for (const [index, response] of responses.entries()) {
assert.equal(response.status, 200);
const payload = await response.json();
assert.equal(payload.choices[0].message.content, `Completed with ${pair[index]} effort.`);
}
assert.equal(bodies.length, pair[0] === pair[1] ? 1 : 2);
assert.deepEqual(bodies.map((b) => b.reasoning_effort).sort(), [...new Set(pair)].sort());
for (const body of bodies) {
assert.equal(body.originModel, undefined);
assert.equal(body.resolvedThinkingEffort, undefined);
assert.equal(body.defaultThinkingEffort, undefined);
assert.equal(body.trustedContext, undefined);
}
} finally {
server.closeAllConnections();
await new Promise<void>((resolve, reject) =>
server.close((error) => (error ? reject(error) : resolve()))
);
}
});
}
for (const pair of [
["high", "none"],
["none", "high"],
["high", "high"],
]) {
test(`overlapping explicit requests ${pair.join("/")} preserve responses over HTTP`, async () => {
const { handleChatCore } = await import("../../open-sse/handlers/chatCore.ts");
const bodies: Record<string, unknown>[] = [];
let firstArrived!: () => void;
const arrival = new Promise<void>((resolve) => {
firstArrived = resolve;
});
const server = createServer(async (req, res) => {
let text = "";
for await (const chunk of req) text += chunk;
const sent = JSON.parse(text);
bodies.push(sent);
firstArrived();
await new Promise((resolve) => setTimeout(resolve, 800));
res.setHeader("Content-Type", "application/json");
res.end(
await buildOpenAIResponse(
`Completed with ${sent.reasoning_effort} effort.`,
"gpt-5.2"
).text()
);
});
server.listen(0, "127.0.0.1");
await once(server, "listening");
const savedFetch = globalThis.fetch;
try {
const address = server.address();
assert.ok(address && typeof address !== "string");
globalThis.fetch = (_input, init) =>
networkFetch(`http://127.0.0.1:${address.port}/v1/chat/completions`, init);
const send = (reasoning_effort: string) =>
handleChatCore({
body: {
model: "gpt-5.2",
stream: false,
temperature: 0,
reasoning_effort,
messages: [{ role: "user", content: "Compute an overlapping explicit example." }],
},
modelInfo: { provider: "openai", model: "gpt-5.2" },
credentials: { apiKey: "fixture-explicit-local" },
log: { debug() {}, info() {}, warn() {}, error() {} },
onCredentialsRefreshed: undefined,
onRequestSuccess: undefined,
onStreamFailure: undefined,
onDisconnect: undefined,
clientRawRequest: undefined,
connectionId: undefined,
userAgent: undefined,
comboName: undefined,
});
const first = send(pair[0]);
await arrival;
const results = await Promise.all([first, send(pair[1])]);
for (const [index, result] of results.entries()) {
const response = result instanceof Response ? result : result.response;
assert.equal(response.status, 200);
const payload = await response.json();
assert.equal(payload.choices[0].message.content, `Completed with ${pair[index]} effort.`);
}
assert.equal(bodies.length, pair[0] === pair[1] ? 1 : 2);
assert.deepEqual(
bodies.map((b) => b.reasoning_effort),
[...new Set(pair)]
);
for (const body of bodies) {
assert.equal(body.temperature, body.reasoning_effort === "high" ? undefined : 0);
assert.equal(body.requestIntent, undefined);
assert.equal(body.trustedContext, undefined);
}
} finally {
globalThis.fetch = savedFetch;
server.closeAllConnections();
await new Promise<void>((resolve, reject) =>
server.close((error) => (error ? reject(error) : resolve()))
);
}
});
}
for (const accountFallback of [false, true]) {
test(`max suffix survives ${accountFallback ? "account fallback" : "same-account retry"} over HTTP`, async () => {
await seedConnection("agnes", { name: "agnes-backup", priority: 2 });
const bodies: Record<string, unknown>[] = [];
const identities = new Map<string, number>();
const accountOrdinals: number[] = [];
const server = createServer(async (req, res) => {
const auth = String(req.headers.authorization || "");
if (!identities.has(auth)) identities.set(auth, identities.size + 1);
accountOrdinals.push(identities.get(auth)!);
let text = "";
for await (const chunk of req) text += chunk;
bodies.push(JSON.parse(text));
res.setHeader("Content-Type", "application/json");
if (accountFallback ? identities.get(auth) === 1 : bodies.length === 1) {
res.writeHead(429);
res.end(JSON.stringify({ error: { message: "Rate limit exceeded" } }));
} else {
res.end(await buildOpenAIResponse("The requested calculation is complete.", MODEL).text());
}
});
server.listen(0, "127.0.0.1");
await once(server, "listening");
try {
const address = server.address();
assert.ok(address && typeof address !== "string");
globalThis.fetch = (_input, init) =>
networkFetch(`http://127.0.0.1:${address.port}/v1/chat/completions`, init);
const response = await handleChat(
buildRequest({
body: {
model: `agnes/${MODEL}-max`,
stream: false,
messages: [{ role: "user", content: "Compute a short example." }],
},
})
);
assert.equal(response.status, 200, await response.text());
assert.ok(!identities.has(""), "all requests must carry a credential");
assert.equal(identities.size, accountFallback ? 2 : 1);
assert.equal(accountOrdinals[0], 1);
assert.equal(accountOrdinals.at(-1), accountFallback ? 2 : 1);
if (!accountFallback) assert.equal(bodies.length, 2);
for (const body of bodies) {
assert.equal(body.model, MODEL);
assert.equal(body.reasoning_effort, "max");
}
} finally {
server.closeAllConnections();
await new Promise<void>((resolve, reject) =>
server.close((error) => (error ? reject(error) : resolve()))
);
}
});
}