pjandro/OmniRoute
1
0
Fork 0
mirror of https://github.com/diegosouzapw/OmniRoute.git synced 2026-09-17 12:22:34 +03:00
Code Issues Packages Projects Releases Wiki Activity

feat: adaptive reasoning effort (auto) — gateway-resolved, per-turn pinned, all harnesses (#13448)

Browse Source 
Merged after a maintainer rework that kept every one of @patrykkopycinski's commits intact — including the two refactors you pushed later (extracting the adaptive-effort wiring out of `chatCore.ts` and reading `x-omniroute-effort` inside the wiring module), which were merged into the rework rather than overwritten.

**What the rework added:** the adaptive-effort wiring is scoped to OpenAI-dispatch requests only (the claim in `docs/routing` was corrected to match), and `defaultReasoningEffort` was widened to accept `auto` explicitly instead of relying on a loose string.

Validated as a combined board first (this PR merged with the 21 siblings of the same wave on the release tip): eslint with the frozen suppressions, typecheck:core, check:open-sse-typecheck, complexity, cognitive-complexity, changelog-integrity, i18n new-key coverage, docs-counts, docs-sync, migration-numbering, provider-consistency and a duplicate-identifier audit all green, plus 176 passing / 0 failing focused node:test cases across the 25 test files the wave touches and the dashboard test under Vitest (2/0). Then re-validated alone on the fresh tip before this merge: conflicts re-resolved, file sizes rebaselined for this PR's own growth, eslint and this PR's focused tests re-run.

Thank you — gateway-resolved, per-turn pinned effort is a real feature, and the header contract makes it usable from every harness.
This commit is contained in:
Patryk Kopyciński
2026-09-16 20:18:36 +02:00
committed by GitHub
parent 421d1ff912
commit 23c5772ccb
10 changed files with 646 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
changelog.d/features/13448-adaptive-reasoning-effort.md Normal file
View File

@@ -0,0 +1 @@
- **feat(reasoning):** adaptive reasoning effort (`auto`) — the gateway resolves the thinking budget per user turn from deterministic request-shape signals (stateless per-turn pin) instead of forwarding a literal `auto`, applied at the gateway pre-translation for any harness (Claude Code, Cursor, Codex, opencode, Hermes) whose request dispatches to an OpenAI Chat-Completions-shaped upstream (`targetFormat === FORMATS.OPENAI` — `reasoning_effort` is an OpenAI-shaped field, so a Claude- or Gemini-targeted request is unaffected). Opt in via `X-OmniRoute-Effort: auto` or a model's `defaultReasoningEffort: "auto"` (now a valid `ModelSpec` value); any explicit client reasoning field always wins ([#13448](https://github.com/diegosouzapw/OmniRoute/pull/13448))

3
config/quality/file-size-baseline.json
View File

@@ -451,6 +451,7 @@
"_rebaseline_2026_09_15_13440_daily_reset_tz": "#13440 rework: open-sse/services/accountFallback.ts 2469->2493 (+24): +6 for the operator-clock-first branch in checkFallbackError non-TPD daily quota (nextConfiguredResetMs leaf lives in dailyQuotaReset.ts, under cap) and +18 from the mandatory lint-staged Prettier pass over pre-existing unformatted lines of the touched file (no logic). executeTargetAttempt.ts 1212->1215 and roundRobinCombo.ts 1205->1208 (+3 each): one import plus the rotation/dailyReset arguments at the existing checkFallbackError call site; the lookup itself is the new comboDailyResetClock.ts leaf (under cap). Covered by tests/unit/daily-reset-tz-threading.test.ts.",
"_rebaseline_2026_09_15_13672_retry_after_provenance": "#13672 rework (opt-in RETRY_AFTER_PROVENANCE_ENABLED): open-sse/services/combo/executeTargetAttempt.ts 1212->1220 (+8) and roundRobinCombo.ts 1205->1210 (+5) at the existing drain-path clone/parse block: capture the already-read body text, log an unreadable hint (debug for a non-JSON page, warn for a failed clone) instead of an empty catch, and one flag-gated prose fallback line; the import grows by the two helpers. Parsing, flag read and the Retry-After/provenance logic live in open-sse/utils/error.ts (under cap). Covered by tests/unit/retry-after-provenance.test.ts (flag off and on).",
"_rebaseline_2026_09_15_13439_protected_priority_stop_status": "#13439 rework (opt-in PROTECTED_PRIORITY_INFRA_502_ENABLED): open-sse/services/combo/executeTargetAttempt.ts 1212->1217 (+5): two import lines for the new protectedPriorityStopStatus.ts leaf (where the provably-non-quota cause list and the flag read live) and the predictive_ttft cause argument at the existing stopProtectedPriorityTarget call, which Prettier splits over three lines. Covered by tests/unit/combo/protected-priority-stop-status-13439.test.ts (every stop cause, flag off and on).",
"_rebaseline_2026_09_16_13448_adaptive_effort_targetformat_gate": "#13448 rework: open-sse/handlers/chatCore.ts 6142->6156 (+14, PR's own growth: the X-OmniRoute-Effort header capture near THINKING_MARKER_HEADER and the wireAdaptiveEffort(translatedBody, {...}) call site right after applyDefaultReasoningEffort, plus this rework's +1 targetFormat argument at that same call). The targetFormat gate itself (ctx.targetFormat !== FORMATS.OPENAI short-circuit) lives in the non-frozen open-sse/handlers/chatCore/adaptiveEffortWiring.ts leaf, not here -- irreducible call-site wiring at the existing post-translation reasoning-normalization chokepoint. Covered by tests/unit/adaptive-effort-wiring.test.ts (targetFormat gate, red-on-tip) and tests/unit/adaptive-effort-model-default-13448.test.ts.",
"_rebaseline_pr1043_minimax_tts": "Upstream port decolua/9router#1043 (toanalien) own growth: audioSpeech.ts 965->1061 (+96). Adds MiniMax T2A v2 TTS dispatch (handleMinimaxSpeech + hexToBytes helper) — provider entry was already in audioRegistry (format: minimax-tts) but no handler existed, falling through to the OpenAI-compatible default that fails (T2A has custom shape + hex-encoded audio + base_resp envelope). New branch sits next to the other inline provider branches (xiaomi-mimo, coqui, tortoise, aws-polly) — extracting would just create indirection. Covered by tests/unit/minimax-tts-1043.test.ts (3 tests, GREEN: success, base_resp error, invalid-hex).",
"_rebaseline_pr4592_exclude_exhausted_auto": "Reconcile #4592 already-merged growth: combo.ts 2991->3036 (+45, terminal-status quota-cutoff exclusion in buildAutoCandidates + opt-in gate). Fast-gate PR->release does not run check:file-size.",
"open-sse/executors/antigravity.ts": 1665,
@@ -459,7 +460,7 @@
"open-sse/executors/codex.ts": 1505,
"open-sse/executors/cursor.ts": 1808,
"open-sse/executors/muse-spark-web.ts": 1405,
"open-sse/handlers/chatCore.ts": 6146,
"open-sse/handlers/chatCore.ts": 6156,
"open-sse/handlers/imageGeneration.ts": 3293,
"open-sse/handlers/search.ts": 1789,
"open-sse/mcp-server/schemas/tools.ts": 1621,

11
docs/routing/AUTO-COMBO.md
View File

@@ -252,11 +252,12 @@ combo's stored config. These apply only to the `auto` strategy and only for the
that carries them; the combo's saved `modePack`/`budgetCap`/`budgetFallback` are used
when the header is absent.
| Header | Accepts | Effect |
| :---------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `X-OmniRoute-Mode` | a preset alias (`fast`, `balanced`, `quality`, `cheap`, `reliable`, `offline`) or a raw pack name (`ship-fast`, `cost-saver`, `quality-first`, `offline-friendly`, `reliability-first`) | Overrides the scoring weights for this request. `balanced`/`default` force the default weights (no pack). Unknown values are ignored (config preserved). |
| `X-OmniRoute-Budget` | a positive number (max USD per request) | Hard cost ceiling: candidates whose estimated cost exceeds it are filtered before selection. What happens when **every** candidate exceeds it is controlled by `X-OmniRoute-Budget-Fallback` below. |
| `X-OmniRoute-Budget-Fallback` | `cheapest` (default, aliases: `cheapest-viable`, `soft`) or `strict` (aliases: `block`, `hard`) | `cheapest`: falls back to the globally cheapest candidate even though it still exceeds the cap (legacy behavior). `strict`: refuses to select — the request fails fast with `HTTP 402` instead of silently overspending. Unknown values are ignored. |
| Header | Accepts | Effect |
| :---------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `X-OmniRoute-Mode` | a preset alias (`fast`, `balanced`, `quality`, `cheap`, `reliable`, `offline`) or a raw pack name (`ship-fast`, `cost-saver`, `quality-first`, `offline-friendly`, `reliability-first`) | Overrides the scoring weights for this request. `balanced`/`default` force the default weights (no pack). Unknown values are ignored (config preserved). |
| `X-OmniRoute-Budget` | a positive number (max USD per request) | Hard cost ceiling: candidates whose estimated cost exceeds it are filtered before selection. What happens when **every** candidate exceeds it is controlled by `X-OmniRoute-Budget-Fallback` below. |
| `X-OmniRoute-Budget-Fallback` | `cheapest` (default, aliases: `cheapest-viable`, `soft`) or `strict` (aliases: `block`, `hard`) | `cheapest`: falls back to the globally cheapest candidate even though it still exceeds the cap (legacy behavior). `strict`: refuses to select — the request fails fast with `HTTP 402` instead of silently overspending. Unknown values are ignored. |
| `X-OmniRoute-Effort` | `auto` (other values reserved) | Adaptive thinking budget: when the request carries **no** reasoning field of any shape (`reasoning_effort`, `reasoning`, `thinking`), the gateway resolves `auto` to `low`/`medium`/`high` from deterministic request-shape signals (last-user-message length, context size up to the last user message, prior tool results, tool-loop depth). Signals are scoped to the current turn — everything after the last user message is ignored — so every request in a tool loop resolves to the same level (stateless per-turn pin, no session state, no mid-loop escalation that would break upstream prompt-cache prefixes). An explicit client reasoning field always wins. Scoped to requests whose upstream dispatch resolves to the OpenAI Chat Completions shape (`targetFormat === FORMATS.OPENAI`) — `reasoning_effort` is an OpenAI-shaped field, so the header is a no-op on a Claude- or Gemini-targeted request (see `open-sse/handlers/chatCore/adaptiveEffortWiring.ts`). |
```bash
# Force the fastest profile, cap this request at $0.05, and hard-block instead of overspending

6
open-sse/handlers/chatCore.ts
View File

@@ -170,6 +170,7 @@ import {
import { shouldUseMidConversationSystem } from "../executors/claudeIdentity.ts";
import { normalizeClaudeHaikuConstraints } from "../services/claudeHaikuConstraints.ts";
import { applyDefaultReasoningEffort } from "../services/defaultReasoningEffort.ts";
import { wireAdaptiveEffort } from "./chatCore/adaptiveEffortWiring.ts";
import { echoModelInObject } from "../services/responseModelEcho.ts";
import {
stripGpt5SamplingWhenReasoning,
@@ -2718,6 +2719,11 @@ export async function handleChatCore({
(modelInfo as { defaultThinkingEffort?: string })?.defaultThinkingEffort
);
}
translatedBody = wireAdaptiveEffort(translatedBody, {
rawBody: body,
clientRawRequest,
targetFormat,
});
}
// Xiaomi MiMo controls reasoning ONLY via `thinking:{type:"enabled"|"disabled"}` and

72
open-sse/handlers/chatCore/adaptiveEffortWiring.ts Normal file
View File

@@ -0,0 +1,72 @@
// Adaptive reasoning-effort wiring (#13448), extracted from chatCore.ts so the
// frozen main file does not grow (file-size gate: chatCore.ts cannot grow).
// Semantics live in open-sse/services/adaptiveEffort.ts; this module only
// adapts the chatCore call-site context (headers, raw body, translated body).
//
// Runs AFTER applyDefaultReasoningEffort so its explicit-value precedence and
// alias-suffix priority are preserved; operates on the pre-translation body so
// source-format differences are handled by the existing translators.
import {
applyAdaptiveEffort,
hasExplicitReasoningField,
isAdaptiveEffort,
type ChatMessageLike,
} from "../../services/adaptiveEffort.ts";
import { FORMATS } from "../../translator/formats.ts";
import { getHeaderValueCaseInsensitive } from "./headers.ts";
export interface AdaptiveEffortContext {
/** Raw (pre-translation) request body, for turn-scoped request-shape signals. */
rawBody: { messages?: ChatMessageLike[] | undefined } | undefined;
/** Incoming client request, used to read the x-omniroute-effort header. */
clientRawRequest?: { headers?: unknown } | undefined;
/** Explicit header value, if already extracted by the caller. */
headerEffort?: string | null | undefined;
/**
* Resolved upstream dispatch format (chatCore.ts's `targetFormat`). `reasoning_effort`
* is an OpenAI Chat-Completions-shaped field: on any other target it either does
* nothing (Claude/Gemini executors read `thinking`/`reasoning.effort` instead and
* never look at it) or, worse, reaches an upstream that rejects unrecognized
* top-level parameters (e.g. Anthropic's Messages API 400s on one). Every other
* reasoning-shape normalization in chatCore.ts (applyDefaultReasoningEffort,
* promoteStrayReasoningEffort for the Responses same-format lane) is scoped the
* same way — wiring must match, or an operator's `X-OmniRoute-Effort: auto` header
* on a Claude/Gemini-targeted request would silently no-op or break the request.
*/
targetFormat: string | undefined;
}
/**
* Resolve "auto" reasoning effort to a concrete level when the request opted in
* (header or ModelSpec.defaultReasoningEffort === "auto") and carries no explicit
* reasoning field. Returns `body` unchanged (same reference) otherwise.
*
* Scoped to `FORMATS.OPENAI` dispatch — see {@link AdaptiveEffortContext.targetFormat}.
*/
export function wireAdaptiveEffort<T extends Record<string, unknown>>(
body: T,
ctx: AdaptiveEffortContext
): T {
if (ctx.targetFormat !== FORMATS.OPENAI) return body;
// Lever: applyDefaultReasoningEffort may have just injected the literal
// "auto" from ModelSpec.defaultReasoningEffort — that is an opt-in marker,
// not a wire value, so it must NOT count as an explicit client field (it
// would otherwise short-circuit the guard below and ship "auto" upstream).
const modelDefaultAuto = isAdaptiveEffort(body.reasoning_effort);
if (!modelDefaultAuto && hasExplicitReasoningField(body)) return body;
const headerEffort =
ctx.headerEffort !== undefined
? ctx.headerEffort
: getHeaderValueCaseInsensitive(
ctx.clientRawRequest?.headers as Record<string, unknown> | Headers | null | undefined,
"x-omniroute-effort"
);
if (!modelDefaultAuto && !isAdaptiveEffort(headerEffort)) return body;
const stripped = modelDefaultAuto ? { ...body } : body;
if (modelDefaultAuto) delete (stripped as Record<string, unknown>).reasoning_effort;
return applyAdaptiveEffort(stripped, {
messages: ctx.rawBody?.messages,
headerEffort: headerEffort ?? null,
modelDefaultEffort: modelDefaultAuto ? "auto" : null,
}) as T;
}

137
open-sse/services/adaptiveEffort.ts Normal file
View File

@@ -0,0 +1,137 @@
// Adaptive reasoning effort — the OmniRoute-side counterpart of Hermes'
// `effort: "auto"` (NousResearch/hermes-agent#109044). One implementation at
// the gateway covers every harness (Claude Code, Cursor, Codex, opencode,
// Hermes) because the full request body passes through here before any
// provider translation.
//
// Semantics:
// - Effort is resolved from deterministic request-shape signals only — no LLM
// call, no judgment gate. Three bands (low / medium / high) gate a thinking
// budget, not a model-routing decision.
// - STATELESS PER-TURN PIN: signals are computed ONLY from the last user
// message and everything BEFORE it. Tool results after the last user
// message are ignored, so every request of the same user turn — including
// mid-tool-loop requests — resolves to the SAME level. This reproduces
// Hermes' stateful per-turn pin deterministically, without stored state,
// and never escalates mid-loop (which would change the reasoning config
// between requests and cost a cold prompt-cache prefix write on
// cache-sensitive upstreams).
//
// Priority (highest first), all off-by-default:
// 1. Explicit client reasoning field of any shape — always wins; no-op.
// 2. `X-OmniRoute-Effort: auto` request header (per-request opt-in; mirrors
// the #6023/#6024/#6025 `X-OmniRoute-Mode`/`-Budget` controls pattern).
// 3. `ModelSpec.defaultReasoningEffort: "auto"` (per-model opt-in, #6879).
import { estimateMessageTokens } from "./specificityRules";
export const ADAPTIVE_EFFORT = "auto";
// Thresholds mirror the Hermes resolver (agent/reasoning_effort.py): three
// coarse deterministic bands. A near-miss costs a slightly over/under-thought
// answer, not a wrong route, so they stay coarse until call-log data says
// otherwise.
const TRIVIAL_USER_CHARS = 160;
const TRIVIAL_CTX_TOKENS = 4000;
const HEAVY_CTX_TOKENS = 60000;
const HEAVY_TOOL_RESULTS = 6;
const HEAVY_USER_CHARS = 4000;
export type ChatMessageLike = { role?: unknown; content?: unknown };
type EffortLevel = "low" | "medium" | "high";
function isString(v: unknown): v is string {
return typeof v === "string";
}
function lastUserMessageIndex(messages: ChatMessageLike[]): number {
let last = -1;
for (let i = 0; i < messages.length; i++) {
if (messages[i]?.role === "user") last = i;
}
return last;
}
function messageTextChars(content: unknown): number {
if (isString(content)) return content.length;
if (Array.isArray(content)) {
let sum = 0;
for (const part of content) {
const text = (part as { text?: unknown })?.text;
if (isString(text)) sum += text.length;
}
return sum;
}
return 0;
}
function countRole(messages: ChatMessageLike[], role: string, from: number, to: number): number {
let n = 0;
for (let i = from; i < to; i++) {
if (messages[i]?.role === role) n++;
}
return n;
}
function toolLoopDepthAfter(messages: ChatMessageLike[], boundary: number): number {
let n = 0;
for (let i = boundary + 1; i < messages.length; i++) {
const msg = messages[i];
if (msg?.role === "assistant" && (msg as { tool_calls?: unknown }).tool_calls) n++;
}
return n;
}
export function resolveAdaptiveEffort(messages: ChatMessageLike[] | undefined | null): EffortLevel {
const msgs = Array.isArray(messages) ? messages : [];
if (msgs.length === 0) return "medium"; // no signals at all → balanced band, never cheap-by-default
const boundary = lastUserMessageIndex(msgs);
const upToTurn = boundary >= 0 ? msgs.slice(0, boundary + 1) : msgs;
const userChars = boundary >= 0 ? messageTextChars(msgs[boundary].content) : 0;
const estCtxTokens = estimateMessageTokens(upToTurn as Array<{ content?: unknown }>);
const toolResults = boundary >= 0 ? countRole(msgs, "tool", 0, boundary) : 0;
const turnDepth = boundary >= 0 ? toolLoopDepthAfter(msgs, boundary) + 1 : 1;
const trivial =
userChars <= TRIVIAL_USER_CHARS &&
estCtxTokens <= TRIVIAL_CTX_TOKENS &&
toolResults === 0 &&
turnDepth <= 1;
if (trivial) return "low";
const heavy =
estCtxTokens >= HEAVY_CTX_TOKENS ||
toolResults >= HEAVY_TOOL_RESULTS ||
userChars >= HEAVY_USER_CHARS;
if (heavy) return "high";
return "medium";
}
export function isAdaptiveEffort(value: unknown): boolean {
return isString(value) && value.trim().toLowerCase() === ADAPTIVE_EFFORT;
}
export function hasExplicitReasoningField(body: Record<string, unknown>): boolean {
return (
body.reasoning_effort !== undefined ||
body.reasoning !== undefined ||
body.thinking !== undefined
);
}
export function applyAdaptiveEffort<T extends Record<string, unknown>>(
body: T,
opts: {
messages?: ChatMessageLike[] | undefined | null;
headerEffort?: unknown;
modelDefaultEffort?: string | null;
}
): T {
if (!body || typeof body !== "object") return body;
if (hasExplicitReasoningField(body)) return body;
const headerAuto = isAdaptiveEffort(opts.headerEffort);
const defaultAuto = opts.modelDefaultEffort != null && isAdaptiveEffort(opts.modelDefaultEffort);
if (!headerAuto && !defaultAuto) return body;
const level = resolveAdaptiveEffort(
opts.messages ?? (body.messages as ChatMessageLike[] | undefined)
);
return { ...body, reasoning_effort: level };
}

12
src/shared/constants/modelSpecs.ts
View File

@@ -48,7 +48,17 @@ export interface ModelSpec {
// operator strip-by-default a thinks-by-default model (measured: gemini-flash-lite
// burns ~277 reasoning tokens on a plain request; `reasoning_effort:"none"` → 0)
// without patching every client. See open-sse/services/defaultReasoningEffort.ts.
defaultReasoningEffort?: "none" | "low" | "medium" | "high";
//
// `"auto"` (#13448) is the per-model opt-in into adaptive reasoning effort: the
// literal value is injected here exactly like any other level, then
// chatCore/adaptiveEffortWiring.ts's wireAdaptiveEffort() recognizes it as an
// opt-in marker (never forwarded upstream verbatim) and resolves it to a
// concrete low/medium/high from the turn's request-shape signals. Without
// "auto" in this union, no operator could configure the per-model opt-in
// through the typed catalog at all -- open-sse/services/adaptiveEffort.ts's
// priority #3 and the wiring's modelDefaultAuto branch were unreachable
// except by a test constructing the body literal directly.
defaultReasoningEffort?: "none" | "low" | "medium" | "high" | "auto";
}
const BEDROCK_CLAUDE_ALIASES = (...modelIds: string[]) => [

97
tests/unit/adaptive-effort-model-default-13448.test.ts Normal file
View File

@@ -0,0 +1,97 @@
/**
* #13448 rework — the per-model opt-in path (`ModelSpec.defaultReasoningEffort:
* "auto"`) was unreachable through the real, type-checked catalog: the field's
* type union was `"none" | "low" | "medium" | "high"`, so no operator config in
* providerRegistry.ts (or a fixture like this one) could ever assign `"auto"`
* without a type error. `open-sse/services/adaptiveEffort.ts`'s priority #3 and
* `adaptiveEffortWiring.ts`'s `modelDefaultAuto` branch were only ever exercised
* by tests that constructed the post-injection body literal directly
* (`{ reasoning_effort: "auto" }`), bypassing the type entirely.
*
* These tests exercise the REAL two-function pipeline end to end, starting
* from a typed `MODEL_SPECS` entry (no `as any`, no literal shortcut):
* MODEL_SPECS[id].defaultReasoningEffort === "auto"
* -> applyDefaultReasoningEffort() injects the literal "auto"
* -> wireAdaptiveEffort() recognizes it as an opt-in marker and resolves
* it to a concrete low/medium/high (OpenAI dispatch only, #13448 rework
* -- see adaptive-effort-wiring.test.ts for the targetFormat gate).
*/
import test from "node:test";
import assert from "node:assert/strict";
import { applyDefaultReasoningEffort } from "../../open-sse/services/defaultReasoningEffort.ts";
import { wireAdaptiveEffort } from "../../open-sse/handlers/chatCore/adaptiveEffortWiring.ts";
import { FORMATS } from "../../open-sse/translator/formats.ts";
import { MODEL_SPECS, type ModelSpec } from "../../src/shared/constants/modelSpecs.ts";
const FIXTURE_MODEL_ID = "__test_13448_model_default_auto__";
// Typed assignment through the real ModelSpec union -- this line alone would
// be a TypeScript error on the pristine tip (`Type '"auto"' is not assignable
// to type '"none" | "low" | "medium" | "high"'`), which is exactly the
// unreachability defect: verify with `npm run -s typecheck:core`.
const fixtureSpec: ModelSpec = { defaultReasoningEffort: "auto" };
test.before(() => {
MODEL_SPECS[FIXTURE_MODEL_ID] = fixtureSpec;
});
test.after(() => {
delete MODEL_SPECS[FIXTURE_MODEL_ID];
});
const HEAVY = "x".repeat(20000);
const trivialMsgs = [{ role: "user", content: "list the files" }];
const heavyMsgs = [{ role: "user", content: HEAVY }];
test("a real ModelSpec.defaultReasoningEffort:'auto' entry is injected as the literal marker", () => {
const body = { model: FIXTURE_MODEL_ID, messages: [] };
const result = applyDefaultReasoningEffort(body, FIXTURE_MODEL_ID);
assert.equal(result.reasoning_effort, "auto");
});
test("end to end: the typed model-default 'auto' resolves to a concrete low level on OpenAI dispatch", () => {
const rawBody = { model: FIXTURE_MODEL_ID, messages: trivialMsgs };
const afterDefault = applyDefaultReasoningEffort(rawBody, FIXTURE_MODEL_ID);
const wired = wireAdaptiveEffort(afterDefault, {
rawBody,
headerEffort: null,
targetFormat: FORMATS.OPENAI,
});
assert.notEqual(wired.reasoning_effort, "auto");
assert.equal(wired.reasoning_effort, "low");
});
test("end to end: the typed model-default 'auto' resolves to 'high' on a heavy turn", () => {
const rawBody = { model: FIXTURE_MODEL_ID, messages: heavyMsgs };
const afterDefault = applyDefaultReasoningEffort(rawBody, FIXTURE_MODEL_ID);
const wired = wireAdaptiveEffort(afterDefault, {
rawBody,
headerEffort: null,
targetFormat: FORMATS.OPENAI,
});
assert.equal(wired.reasoning_effort, "high");
});
test("end to end: an explicit client reasoning_effort still wins over the typed model default", () => {
const rawBody = { model: FIXTURE_MODEL_ID, messages: heavyMsgs, reasoning_effort: "medium" };
const afterDefault = applyDefaultReasoningEffort(rawBody, FIXTURE_MODEL_ID);
assert.equal(afterDefault.reasoning_effort, "medium", "no-op: explicit field already present");
const wired = wireAdaptiveEffort(afterDefault, {
rawBody,
headerEffort: null,
targetFormat: FORMATS.OPENAI,
});
assert.equal(wired.reasoning_effort, "medium");
});
test("end to end: on a non-OpenAI target the typed model-default marker is left as literal 'auto' (never resolved, never forwarded resolved)", () => {
const rawBody = { model: FIXTURE_MODEL_ID, messages: heavyMsgs };
const afterDefault = applyDefaultReasoningEffort(rawBody, FIXTURE_MODEL_ID);
const wired = wireAdaptiveEffort(afterDefault, {
rawBody,
headerEffort: null,
targetFormat: FORMATS.CLAUDE,
});
assert.equal(wired, afterDefault, "same reference: wireAdaptiveEffort must no-op off OpenAI");
assert.equal(wired.reasoning_effort, "auto");
});

225
tests/unit/adaptive-effort-wiring.test.ts Normal file
View File

@@ -0,0 +1,225 @@
// chatCore adaptive-effort wiring tests (#13448).
//
// The wiring module is the chatCore call-site adapter extracted from
// chatCore.ts (file-size gate: that file cannot grow). The service-level
// tests in adaptive-effort.test.ts cover resolution semantics; THESE tests
// cover the adapter's own decisions, which are invisible to the service:
// - explicit client reasoning fields are never overwritten (precedence),
// - the literal "auto" injected by ModelSpec.defaultReasoningEffort is an
// opt-in marker and must be stripped before resolution, not sent upstream,
// - the x-omniroute-effort header opts in independently (read inside the
// module from `clientRawRequest.headers`, or passed pre-extracted),
// - a non-opted-in body is returned untouched (same reference),
// - the whole wiring is scoped to OpenAI Chat-Completions dispatch.
import { test } from "node:test";
import assert from "node:assert/strict";
import { wireAdaptiveEffort } from "@omniroute/open-sse/handlers/chatCore/adaptiveEffortWiring.ts";
import { FORMATS } from "@omniroute/open-sse/translator/formats.ts";
const HEAVY = "x".repeat(20000);
const trivialMsgs = [{ role: "user", content: "list the files" }];
const heavyMsgs = [{ role: "user", content: HEAVY }];
test("explicit reasoning_effort is never overwritten by adaptive wiring", () => {
const body = { model: "m", reasoning_effort: "low" };
const out = wireAdaptiveEffort(body, {
rawBody: { messages: heavyMsgs },
headerEffort: "auto",
targetFormat: FORMATS.OPENAI,
});
assert.equal(out.reasoning_effort, "low");
});
test("explicit reasoning object is never overwritten", () => {
const body = { model: "m", reasoning: { effort: "high" } };
const out = wireAdaptiveEffort(body, {
rawBody: { messages: trivialMsgs },
headerEffort: "auto",
targetFormat: FORMATS.OPENAI,
});
assert.deepEqual(out.reasoning, { effort: "high" });
assert.equal(out.reasoning_effort, undefined);
});
test("explicit thinking field is never overwritten", () => {
const body = { model: "m", thinking: { type: "enabled" } };
const out = wireAdaptiveEffort(body, {
rawBody: { messages: heavyMsgs },
headerEffort: "auto",
targetFormat: FORMATS.OPENAI,
});
assert.deepEqual(out.thinking, { type: "enabled" });
assert.equal(out.reasoning_effort, undefined);
});
test("model-default 'auto' marker is resolved, never sent upstream verbatim", () => {
const body = { model: "m", reasoning_effort: "auto" };
const out = wireAdaptiveEffort(body, {
rawBody: { messages: trivialMsgs },
headerEffort: null,
targetFormat: FORMATS.OPENAI,
});
assert.notEqual(out.reasoning_effort, "auto");
assert.equal(out.reasoning_effort, "low");
});
test("model-default 'auto' resolves high on heavy turns", () => {
const body = { model: "m", reasoning_effort: "auto" };
const out = wireAdaptiveEffort(body, {
rawBody: { messages: heavyMsgs },
headerEffort: null,
targetFormat: FORMATS.OPENAI,
});
assert.equal(out.reasoning_effort, "high");
});
test("header opt-in resolves from the raw (pre-translation) body messages", () => {
const body = { model: "m" };
const out = wireAdaptiveEffort(body, {
rawBody: { messages: heavyMsgs },
headerEffort: "auto",
targetFormat: FORMATS.OPENAI,
});
assert.equal(out.reasoning_effort, "high");
});
test("no opt-in leaves the body untouched (same reference)", () => {
const body = { model: "m" };
const out = wireAdaptiveEffort(body, {
rawBody: { messages: heavyMsgs },
headerEffort: null,
targetFormat: FORMATS.OPENAI,
});
assert.equal(out, body);
assert.equal(out.reasoning_effort, undefined);
});
test("missing rawBody does not throw", () => {
const body = { model: "m", reasoning_effort: "auto" };
const out = wireAdaptiveEffort(body, {
rawBody: undefined,
headerEffort: null,
targetFormat: FORMATS.OPENAI,
});
assert.ok(["low", "medium", "high"].includes(out.reasoning_effort as string));
});
// The x-omniroute-effort header is read INSIDE the module from the incoming
// client request (chatCore.ts passes `clientRawRequest` through untouched), so
// the call site does not need its own header extraction.
test("header is read from clientRawRequest.headers (plain record, case-insensitive)", () => {
const body = { model: "m" };
const out = wireAdaptiveEffort(body, {
rawBody: { messages: heavyMsgs },
clientRawRequest: { headers: { "X-OmniRoute-Effort": "auto" } },
targetFormat: FORMATS.OPENAI,
});
assert.equal(out.reasoning_effort, "high");
});
test("header is read from clientRawRequest.headers (Headers instance)", () => {
const body = { model: "m" };
const out = wireAdaptiveEffort(body, {
rawBody: { messages: trivialMsgs },
clientRawRequest: { headers: new Headers({ "x-omniroute-effort": "auto" }) },
targetFormat: FORMATS.OPENAI,
});
assert.equal(out.reasoning_effort, "low");
});
test("clientRawRequest without the header (or without headers at all) is not an opt-in", () => {
const body = { model: "m" };
const noHeader = wireAdaptiveEffort(body, {
rawBody: { messages: heavyMsgs },
clientRawRequest: { headers: { "user-agent": "x" } },
targetFormat: FORMATS.OPENAI,
});
assert.equal(noHeader, body);
const noHeaders = wireAdaptiveEffort(body, {
rawBody: { messages: heavyMsgs },
clientRawRequest: {},
targetFormat: FORMATS.OPENAI,
});
assert.equal(noHeaders, body);
const noRequest = wireAdaptiveEffort(body, {
rawBody: { messages: heavyMsgs },
targetFormat: FORMATS.OPENAI,
});
assert.equal(noRequest, body);
});
test("a pre-extracted headerEffort takes precedence over clientRawRequest.headers", () => {
const body = { model: "m" };
const out = wireAdaptiveEffort(body, {
rawBody: { messages: heavyMsgs },
headerEffort: null,
clientRawRequest: { headers: { "x-omniroute-effort": "auto" } },
targetFormat: FORMATS.OPENAI,
});
assert.equal(out, body, "explicit null means the caller already decided: no opt-in");
});
// #13448 rework: the field wireAdaptiveEffort injects (`reasoning_effort`) is an
// OpenAI Chat-Completions-shaped field. On any other dispatch format it is either
// inert (Claude/Gemini read `thinking`/`reasoning.effort` instead) or actively
// harmful (Anthropic's Messages API 400s on an unrecognized top-level parameter).
// Every sibling reasoning-shape normalization in chatCore.ts is scoped to
// `FORMATS.OPENAI` the same way (applyDefaultReasoningEffort,
// promoteStrayReasoningEffort's same-format Responses lane) -- wiring must match.
test("header opt-in is a no-op on a Claude-targeted dispatch (body returned unchanged)", () => {
const body = { model: "m" };
const out = wireAdaptiveEffort(body, {
rawBody: { messages: heavyMsgs },
headerEffort: "auto",
targetFormat: FORMATS.CLAUDE,
});
assert.equal(out, body, "must be the exact same reference -- no reasoning_effort injected");
assert.equal(out.reasoning_effort, undefined);
});
test("header opt-in is a no-op on a Gemini-targeted dispatch", () => {
const body = { model: "m" };
const out = wireAdaptiveEffort(body, {
rawBody: { messages: heavyMsgs },
headerEffort: "auto",
targetFormat: FORMATS.GEMINI,
});
assert.equal(out, body);
assert.equal(out.reasoning_effort, undefined);
});
test("header read from clientRawRequest is also a no-op on a non-OpenAI target", () => {
const body = { model: "m" };
const out = wireAdaptiveEffort(body, {
rawBody: { messages: heavyMsgs },
clientRawRequest: { headers: { "x-omniroute-effort": "auto" } },
targetFormat: FORMATS.CLAUDE,
});
assert.equal(out, body);
assert.equal(out.reasoning_effort, undefined);
});
test("model-default 'auto' marker is left untouched (not stripped, not resolved) on a non-OpenAI target", () => {
// Guards against a partial fix that strips the "auto" marker before the
// targetFormat check -- on a non-OpenAI target the body (including any stray
// literal "auto") must be untouched, since it was never OmniRoute's own
// injection to interpret on that dispatch shape.
const body = { model: "m", reasoning_effort: "auto" };
const out = wireAdaptiveEffort(body, {
rawBody: { messages: heavyMsgs },
headerEffort: null,
targetFormat: FORMATS.CLAUDE,
});
assert.equal(out, body);
assert.equal(out.reasoning_effort, "auto");
});
test("targetFormat undefined (e.g. an uncovered call site) also no-ops -- fail closed", () => {
const body = { model: "m" };
const out = wireAdaptiveEffort(body, {
rawBody: { messages: heavyMsgs },
headerEffort: "auto",
targetFormat: undefined,
});
assert.equal(out, body);
});

89
tests/unit/adaptive-effort.test.ts Normal file
View File

@@ -0,0 +1,89 @@
// Adaptive effort tests — mirrors the Hermes contract (hermes-agent#109044):
// trivial→low, heavy→high, mid→medium; explicit client effort wins; the
// stateless per-turn pin ignores post-last-user tool traffic.
import { test } from "node:test";
import assert from "node:assert/strict";
import {
resolveAdaptiveEffort,
applyAdaptiveEffort,
isAdaptiveEffort,
hasExplicitReasoningField,
} from "@omniroute/open-sse/services/adaptiveEffort.ts";
function msg(
role: string,
content: string | unknown[]
): { role: string; content: string | unknown[] } {
return { role, content };
}
test("trivial ask resolves low", () => {
const level = resolveAdaptiveEffort([msg("user", "list the files")]);
assert.equal(level, "low");
});
test("heavy context resolves high", () => {
const level = resolveAdaptiveEffort([
msg("user", "long enough"),
msg("assistant", "x".repeat(40)),
msg("user", "continue " + "deep work ".repeat(400)),
]);
assert.equal(level, "high");
});
test("trivial ask inside big prior context is NOT low", () => {
const level = resolveAdaptiveEffort([
msg("user", "first ask that is long enough to not be trivial itself " + "pad ".repeat(60)),
msg("assistant", "answer " + "y".repeat(30000)),
msg("user", "short follow-up"),
]);
assert.equal(level, "medium");
});
test("stateless pin: mid-tool-loop request resolves same as turn start", () => {
const userTurn = [msg("user", "fix the failing test")];
const turnStart = resolveAdaptiveEffort(userTurn);
const midLoop = resolveAdaptiveEffort([
...userTurn,
msg("assistant", "checking"),
msg("tool", "z".repeat(5000)),
msg("assistant", "checking more"),
msg("tool", "z".repeat(5000)),
]);
// Mid-loop: same turn → same level (pin), even though raw context grew.
assert.equal(midLoop, turnStart);
});
test("explicit effort wins over auto", () => {
const body = { messages: [msg("user", "hello there")], reasoning_effort: "high" };
const out = applyAdaptiveEffort(body, { headerEffort: "auto", modelDefaultEffort: "auto" });
assert.equal(out.reasoning_effort, "high");
assert.equal(hasExplicitReasoningField(body), true);
});
test("header auto on trivial ask injects low", () => {
const body: Record<string, unknown> = { messages: [msg("user", "hi")] };
const out = applyAdaptiveEffort(body, { headerEffort: "auto" });
assert.equal(out.reasoning_effort, "low");
});
test("model default auto injects resolved level", () => {
const out = applyAdaptiveEffort({ messages: [msg("user", "hello")] } as Record<string, unknown>, {
modelDefaultEffort: "auto",
});
assert.equal(out.reasoning_effort, "low");
assert.equal(isAdaptiveEffort("auto"), true);
assert.equal(isAdaptiveEffort("AUTO "), true);
});
test("no auto opt-in → unchanged reference", () => {
const body = { messages: [msg("user", "hello")] };
const out = applyAdaptiveEffort(body, {});
assert.equal(out, body);
});
test("empty messages resolves medium (safe default)", () => {
assert.equal(resolveAdaptiveEffort([]), "medium");
assert.equal(resolveAdaptiveEffort(undefined), "medium");
assert.equal(resolveAdaptiveEffort(null), "medium");
});