mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-07-26 09:52:11 +03:00
Auto-combo transparency + budget controls: the engine's budgetCap enforcement always degraded to the globally cheapest candidate when every candidate exceeded the cap - silently overspending instead of respecting the cap. - engine.ts: budgetFallback "cheapest" (default, legacy) | "strict" (BudgetExceededError when no candidate fits budgetCap) - requestControls.ts: X-OmniRoute-Budget-Fallback header + resolveRequestAutoControls() consolidating mode/budget/fallback parsing - resolveAutoStrategy.ts / autoConfig.ts: thread combo-level config.budgetFallback and catch BudgetExceededError into an HTTP 402 - chat.ts: switch to the consolidated resolveRequestAutoControls() helper (net line reduction, stays under the frozen file-size baseline) Regression guard: tests/unit/auto-combo-budget-fallback-3470.test.ts
This commit is contained in:
committed by
GitHub
parent
d96bd80343
commit
5e1a325e72
@@ -12,6 +12,7 @@ _Living section — bullets land here as PRs merge into `release/v3.8.47` (paral
|
||||
|
||||
### ✨ New Features
|
||||
|
||||
- **feat(combo):** `auto/*` combos gain a strict budget-cap fallback policy — `X-OmniRoute-Budget-Fallback: strict` (or the persisted `config.budgetFallback: "strict"`) makes an over-budget request fail fast with `HTTP 402` instead of the previous silent fallback to the globally cheapest candidate, which could still exceed the cap. The default (`cheapest`) preserves existing behavior. Builds on the existing `X-OmniRoute-Budget`/`X-OmniRoute-Mode` per-request controls (#6023/#6024/#6025), consolidated into `resolveRequestAutoControls()`. Regression guard: `tests/unit/auto-combo-budget-fallback-3470.test.ts`. (#3470)
|
||||
- **Provider/model param filters**: config-driven parameter denylist/allowlist per provider/model with auto-learn from upstream 400s (#6649 — thanks @ThongAccount, closes #6625)
|
||||
- **Per-combo reasoning token buffer toggle**: the combo builder now exposes an explicit checkbox for the `#3587` reasoning-model `max_tokens` buffer, defaulting to the existing enabled behavior, so a combo can opt out without hand-editing raw JSON config (#6702 — thanks @xz-dev)
|
||||
- **feat(dashboard):** 9router-parity **Routing Strategy** settings card on Settings → Routing, plus a per-provider account-routing override on the provider detail page ([#6678](https://github.com/diegosouzapw/OmniRoute/pull/6678)) — surfaces the existing account round-robin / sticky-limit knobs and adds a new combo-level sticky round-robin (`comboStickyRoundRobinLimit`, resolved via `resolveComboStickyRoundRobinLimit()` — per-combo → global combo sticky → account sticky cascade) so combo targets can batch calls per target the same way account fallback already does. A new `providerStrategies` setting (Zod-validated map, `src/shared/validation/settingsSchemas.ts`) lets a specific provider override the global `fallbackStrategy`/`stickyRoundRobinLimit` without touching the account-wide default, wired into `getProviderCredentials()` (`src/sse/services/auth.ts`) ahead of the global fallback. Regression guard: `tests/unit/combo-rr-sticky-9router.test.ts`, `tests/unit/settings-ui-layout-static.test.ts`. (thanks @SeaXen)
|
||||
|
||||
@@ -148,29 +148,33 @@ Notes:
|
||||
- **quality-first** → taskFit 0.37 + stability 0.15 (best model for the task, consistent)
|
||||
- **offline-friendly** → quota 0.37 + health 0.28 (max headroom regardless of speed/cost)
|
||||
|
||||
### Per-Request Controls (headers) — #6023 / #6024 / #6025
|
||||
### Per-Request Controls (headers) — #6023 / #6024 / #6025 / #3470
|
||||
|
||||
An `auto` combo can be steered **per request** via two headers, without mutating the
|
||||
An `auto` combo can be steered **per request** via three headers, without mutating the
|
||||
combo's stored config. These apply only to the `auto` strategy and only for the request
|
||||
that carries them; the combo's saved `modePack`/`budgetCap` are used when the header is
|
||||
absent.
|
||||
that carries them; the combo's saved `modePack`/`budgetCap`/`budgetFallback` are used
|
||||
when the header is absent.
|
||||
|
||||
| Header | Accepts | Effect |
|
||||
| :------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `X-OmniRoute-Mode` | a preset alias (`fast`, `balanced`, `quality`, `cheap`, `reliable`, `offline`) or a raw pack name (`ship-fast`, `cost-saver`, `quality-first`, `offline-friendly`, `reliability-first`) | Overrides the scoring weights for this request. `balanced`/`default` force the default weights (no pack). Unknown values are ignored (config preserved). |
|
||||
| `X-OmniRoute-Budget` | a positive number (max USD per request) | Hard cost ceiling: candidates whose estimated cost exceeds it are filtered before selection, falling back to the cheapest healthy candidate if all exceed. Non-positive/garbage values are ignored. |
|
||||
| Header | Accepts | Effect |
|
||||
| :----------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `X-OmniRoute-Mode` | a preset alias (`fast`, `balanced`, `quality`, `cheap`, `reliable`, `offline`) or a raw pack name (`ship-fast`, `cost-saver`, `quality-first`, `offline-friendly`, `reliability-first`) | Overrides the scoring weights for this request. `balanced`/`default` force the default weights (no pack). Unknown values are ignored (config preserved). |
|
||||
| `X-OmniRoute-Budget` | a positive number (max USD per request) | Hard cost ceiling: candidates whose estimated cost exceeds it are filtered before selection. What happens when **every** candidate exceeds it is controlled by `X-OmniRoute-Budget-Fallback` below. |
|
||||
| `X-OmniRoute-Budget-Fallback` | `cheapest` (default, aliases: `cheapest-viable`, `soft`) or `strict` (aliases: `block`, `hard`) | `cheapest`: falls back to the globally cheapest candidate even though it still exceeds the cap (legacy behavior). `strict`: refuses to select — the request fails fast with `HTTP 402` instead of silently overspending. Unknown values are ignored. |
|
||||
|
||||
```bash
|
||||
# Force the fastest profile and cap this request at $0.05
|
||||
# Force the fastest profile, cap this request at $0.05, and hard-block instead of overspending
|
||||
curl -sS http://localhost:20128/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "X-OmniRoute-Mode: fast" \
|
||||
-H "X-OmniRoute-Budget: 0.05" \
|
||||
-H "X-OmniRoute-Budget-Fallback: strict" \
|
||||
-d '{"model":"auto","messages":[{"role":"user","content":"hi"}]}'
|
||||
```
|
||||
|
||||
Resolution is a pure function (`open-sse/services/autoCombo/requestControls.ts`); the
|
||||
resolved values feed the engine's existing `config.modePack` / `config.budgetCap` inputs.
|
||||
resolved values feed the engine's existing `config.modePack` / `config.budgetCap` /
|
||||
`config.budgetFallback` inputs. A combo's stored `config.budgetFallback` ("strict" |
|
||||
"cheapest") sets the persistent policy; the header overrides it for a single request.
|
||||
|
||||
## All Routing Strategies
|
||||
|
||||
|
||||
@@ -30,11 +30,39 @@ export interface AutoComboConfig {
|
||||
weights: ScoringWeights;
|
||||
modePack?: string;
|
||||
budgetCap?: number; // max cost per request in USD
|
||||
/**
|
||||
* Policy applied when EVERY candidate exceeds `budgetCap` (#3470):
|
||||
* - "cheapest" (default): fall back to the globally cheapest candidate, even
|
||||
* though it still exceeds the cap (existing/legacy behavior).
|
||||
* - "strict": refuse to select — `selectProvider()` throws `BudgetExceededError`
|
||||
* so the caller can surface a clear cost-exceeds-budget response instead of
|
||||
* silently overspending.
|
||||
*/
|
||||
budgetFallback?: "cheapest" | "strict";
|
||||
explorationRate: number; // 0.05 = 5% exploratory
|
||||
/** If set, RouterStrategy name to use for selection ('rules' | 'cost' | 'latency') */
|
||||
routerStrategy?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Thrown by `selectProvider()` when `budgetFallback: "strict"` is set and no
|
||||
* candidate (including the cheapest) fits within `budgetCap` (#3470). Callers
|
||||
* should catch this and surface a cost-exceeds-budget response — never let it
|
||||
* propagate as an unhandled 500.
|
||||
*/
|
||||
export class BudgetExceededError extends Error {
|
||||
constructor(
|
||||
public readonly budgetCap: number,
|
||||
public readonly cheapestCostUsd: number
|
||||
) {
|
||||
super(
|
||||
`No candidate fits within the configured budget cap of $${budgetCap.toFixed(4)} ` +
|
||||
`(cheapest available candidate costs $${cheapestCostUsd.toFixed(4)})`
|
||||
);
|
||||
this.name = "BudgetExceededError";
|
||||
}
|
||||
}
|
||||
|
||||
export interface SelectionResult {
|
||||
provider: string;
|
||||
model: string;
|
||||
@@ -293,6 +321,9 @@ export function selectProvider(
|
||||
const cheapest = [...candidates_].sort(
|
||||
(a, b) => estimatedCostFor(a) - estimatedCostFor(b)
|
||||
)[0];
|
||||
if (config.budgetFallback === "strict") {
|
||||
throw new BudgetExceededError(config.budgetCap, cheapest ? estimatedCostFor(cheapest) : 0);
|
||||
}
|
||||
if (cheapest) selected = cheapest;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -15,4 +15,9 @@ export {
|
||||
export { getTaskFitness, getTaskTypes } from "./taskFitness";
|
||||
export { SelfHealingManager, getSelfHealingManager } from "./selfHealing";
|
||||
export { MODE_PACKS, getModePack, getModePackNames } from "./modePacks";
|
||||
export { selectProvider, type AutoComboConfig, type SelectionResult } from "./engine";
|
||||
export {
|
||||
selectProvider,
|
||||
BudgetExceededError,
|
||||
type AutoComboConfig,
|
||||
type SelectionResult,
|
||||
} from "./engine";
|
||||
|
||||
@@ -1,16 +1,17 @@
|
||||
/**
|
||||
* Per-request Auto-Combo routing controls (#6023 / #6024 / #6025).
|
||||
* Per-request Auto-Combo routing controls (#6023 / #6024 / #6025 / #3470).
|
||||
*
|
||||
* These let a caller steer an `auto` combo on a single request via response-safe
|
||||
* request headers, without changing the combo's stored config:
|
||||
*
|
||||
* X-OmniRoute-Mode: fast | balanced | quality | <raw mode-pack name> (#6024/#6025)
|
||||
* X-OmniRoute-Budget: <max USD per request> (#6023)
|
||||
* X-OmniRoute-Mode: fast | balanced | quality | <raw mode-pack name> (#6024/#6025)
|
||||
* X-OmniRoute-Budget: <max USD per request> (#6023)
|
||||
* X-OmniRoute-Budget-Fallback: cheapest | strict (#3470)
|
||||
*
|
||||
* Both resolvers are pure so they can be unit-tested and reused by the entry
|
||||
* All resolvers are pure so they can be unit-tested and reused by the entry
|
||||
* handler (src/sse/handlers/chat.ts) and the combo router (open-sse/services/combo.ts).
|
||||
* The resolved values feed the auto-combo engine's existing `config.modePack` /
|
||||
* `config.budgetCap` inputs — no engine changes required.
|
||||
* `config.budgetCap` / `config.budgetFallback` inputs.
|
||||
*/
|
||||
|
||||
import { MODE_PACKS } from "./modePacks";
|
||||
@@ -78,3 +79,51 @@ export function parseRequestBudgetCap(input: unknown): number | undefined {
|
||||
if (!Number.isFinite(n) || n <= 0) return undefined;
|
||||
return n;
|
||||
}
|
||||
|
||||
/** Policy applied when every candidate exceeds `budgetCap` — see `AutoComboConfig.budgetFallback`. */
|
||||
export type RequestBudgetFallback = "cheapest" | "strict";
|
||||
|
||||
/**
|
||||
* Parse the `X-OmniRoute-Budget-Fallback` header into a budget-fallback policy override.
|
||||
* Unknown/empty/non-string values return `undefined` so the combo's own stored
|
||||
* `config.budgetFallback` (or the engine default of `"cheapest"`) stays in effect.
|
||||
*/
|
||||
export function parseRequestBudgetFallback(input: unknown): RequestBudgetFallback | undefined {
|
||||
if (typeof input !== "string") return undefined;
|
||||
const key = input.trim().toLowerCase();
|
||||
if (key === "strict" || key === "block" || key === "hard") return "strict";
|
||||
if (key === "cheapest" || key === "cheapest-viable" || key === "soft") return "cheapest";
|
||||
return undefined;
|
||||
}
|
||||
|
||||
/** Aggregated per-request auto-combo overrides resolved from request headers (#3470). */
|
||||
export interface PerRequestAutoControls {
|
||||
mode?: string;
|
||||
budgetCap?: number;
|
||||
budgetFallback?: RequestBudgetFallback;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve all per-request Auto-Combo headers in one pass, returning only the keys
|
||||
* that were actually overridden. Consolidates `resolveRequestModePack()` /
|
||||
* `parseRequestBudgetCap()` / `parseRequestBudgetFallback()` so entry handlers (e.g.
|
||||
* `src/sse/handlers/chat.ts`) can thread a single object into `relayOptions` instead
|
||||
* of repeating the per-header boilerplate for each new control.
|
||||
*/
|
||||
export function resolveRequestAutoControls(headers: {
|
||||
get(name: string): string | null;
|
||||
}): PerRequestAutoControls {
|
||||
const modeHeader = headers.get("x-omniroute-mode")?.trim() || null;
|
||||
const budgetHeader = headers.get("x-omniroute-budget")?.trim() || null;
|
||||
const budgetFallbackHeader = headers.get("x-omniroute-budget-fallback")?.trim() || null;
|
||||
|
||||
const mode = resolveRequestModePack(modeHeader);
|
||||
const budgetCap = parseRequestBudgetCap(budgetHeader);
|
||||
const budgetFallback = parseRequestBudgetFallback(budgetFallbackHeader);
|
||||
|
||||
return {
|
||||
...(mode.override && modeHeader ? { mode: modeHeader } : {}),
|
||||
...(budgetCap !== undefined ? { budgetCap } : {}),
|
||||
...(budgetFallback !== undefined ? { budgetFallback } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -44,6 +44,12 @@ export function parseAutoConfig(combo: ComboLike, eligibleTargets: ResolvedCombo
|
||||
const budgetCap = Number.isFinite(Number(autoConfigSource.budgetCap))
|
||||
? Number(autoConfigSource.budgetCap)
|
||||
: undefined;
|
||||
// #3470: persisted fallback policy for when EVERY candidate exceeds budgetCap.
|
||||
// Any other value (including absent) falls through to the engine's "cheapest" default.
|
||||
const budgetFallback: "strict" | "cheapest" | undefined =
|
||||
autoConfigSource.budgetFallback === "strict" || autoConfigSource.budgetFallback === "cheapest"
|
||||
? (autoConfigSource.budgetFallback as "strict" | "cheapest")
|
||||
: undefined;
|
||||
const modePack =
|
||||
typeof autoConfigSource.modePack === "string" ? autoConfigSource.modePack : undefined;
|
||||
const resetWindowConfig = resolveResetWindowConfig(autoConfigSource);
|
||||
@@ -55,6 +61,7 @@ export function parseAutoConfig(combo: ComboLike, eligibleTargets: ResolvedCombo
|
||||
weights,
|
||||
explorationRate,
|
||||
budgetCap,
|
||||
budgetFallback,
|
||||
modePack,
|
||||
resetWindowConfig,
|
||||
slaPolicy,
|
||||
|
||||
@@ -1,8 +1,12 @@
|
||||
import { unavailableResponse } from "../../utils/error.ts";
|
||||
import { selectProvider as selectAutoProvider } from "../autoCombo/engine.ts";
|
||||
import { errorResponse, unavailableResponse } from "../../utils/error.ts";
|
||||
import {
|
||||
BudgetExceededError,
|
||||
selectProvider as selectAutoProvider,
|
||||
} from "../autoCombo/engine.ts";
|
||||
import {
|
||||
resolveRequestModePack,
|
||||
parseRequestBudgetCap,
|
||||
parseRequestBudgetFallback,
|
||||
} from "../autoCombo/requestControls.ts";
|
||||
import { selectWithStrategy } from "../autoCombo/routerStrategy.ts";
|
||||
import { buildComplexityRoutingHint } from "../autoCombo/complexityRouter";
|
||||
@@ -58,6 +62,8 @@ export interface ResolveAutoStrategyDeps {
|
||||
mode?: string | null;
|
||||
/** Per-request X-OmniRoute-Budget value in USD (#6023). */
|
||||
budgetCap?: number | null;
|
||||
/** Per-request X-OmniRoute-Budget-Fallback value ("cheapest" | "strict") — #3470. */
|
||||
budgetFallback?: "cheapest" | "strict" | null;
|
||||
} | null;
|
||||
resilienceSettings: ResilienceSettings;
|
||||
log: ComboLogger;
|
||||
@@ -157,25 +163,29 @@ export async function resolveAutoStrategyOrder(
|
||||
weights,
|
||||
explorationRate,
|
||||
budgetCap: configBudgetCap,
|
||||
budgetFallback: configBudgetFallback,
|
||||
modePack: configModePack,
|
||||
resetWindowConfig,
|
||||
slaPolicy,
|
||||
} = parseAutoConfig(combo, eligibleTargets);
|
||||
|
||||
// Per-request overrides (#6023 / #6024 / #6025): X-OmniRoute-Budget and
|
||||
// X-OmniRoute-Mode headers (threaded via relayOptions) take precedence over
|
||||
// the combo's stored config for this single request. Unknown/garbage header
|
||||
// values are ignored so the saved config is preserved.
|
||||
// Per-request overrides (#6023 / #6024 / #6025 / #3470): X-OmniRoute-Budget,
|
||||
// X-OmniRoute-Budget-Fallback and X-OmniRoute-Mode headers (threaded via
|
||||
// relayOptions) take precedence over the combo's stored config for this single
|
||||
// request. Unknown/garbage header values are ignored so the saved config is
|
||||
// preserved.
|
||||
const requestBudgetCap = parseRequestBudgetCap(relayOptions?.budgetCap);
|
||||
const budgetCap = requestBudgetCap ?? configBudgetCap;
|
||||
const requestBudgetFallback = parseRequestBudgetFallback(relayOptions?.budgetFallback);
|
||||
const budgetFallback = requestBudgetFallback ?? configBudgetFallback;
|
||||
const requestModePack = resolveRequestModePack(relayOptions?.mode);
|
||||
const modePack = requestModePack.override ? requestModePack.modePack : configModePack;
|
||||
if (requestModePack.override || requestBudgetCap !== undefined) {
|
||||
if (requestModePack.override || requestBudgetCap !== undefined || requestBudgetFallback !== undefined) {
|
||||
log.debug?.(
|
||||
"COMBO",
|
||||
`Auto strategy: per-request controls applied (mode=${
|
||||
requestModePack.override ? (requestModePack.modePack ?? "balanced") : "—"
|
||||
}, budgetCap=${requestBudgetCap ?? "—"})`
|
||||
}, budgetCap=${requestBudgetCap ?? "—"}, budgetFallback=${requestBudgetFallback ?? "—"})`
|
||||
);
|
||||
}
|
||||
|
||||
@@ -256,20 +266,32 @@ export async function resolveAutoStrategyOrder(
|
||||
}
|
||||
|
||||
if (!selectedProvider || !selectedModel) {
|
||||
const selection = selectAutoProvider(
|
||||
{
|
||||
id: combo.id || combo.name,
|
||||
name: combo.name,
|
||||
type: "auto",
|
||||
candidatePool,
|
||||
weights,
|
||||
modePack,
|
||||
budgetCap,
|
||||
explorationRate,
|
||||
},
|
||||
routableCandidates,
|
||||
taskType
|
||||
);
|
||||
let selection;
|
||||
try {
|
||||
selection = selectAutoProvider(
|
||||
{
|
||||
id: combo.id || combo.name,
|
||||
name: combo.name,
|
||||
type: "auto",
|
||||
candidatePool,
|
||||
weights,
|
||||
modePack,
|
||||
budgetCap,
|
||||
budgetFallback,
|
||||
explorationRate,
|
||||
},
|
||||
routableCandidates,
|
||||
taskType
|
||||
);
|
||||
} catch (err) {
|
||||
// #3470: `budgetFallback: "strict"` refuses to select when every candidate
|
||||
// exceeds `budgetCap` — surface a clear cost-exceeds-budget response
|
||||
// instead of letting it propagate as an unhandled 500.
|
||||
if (err instanceof BudgetExceededError) {
|
||||
return { earlyResponse: errorResponse(402, err.message) };
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
selectedProvider = selection.provider;
|
||||
selectedModel = selection.model;
|
||||
selectionReason = `score=${selection.score.toFixed(3)}${selection.isExploration ? " (exploration)" : ""}`;
|
||||
|
||||
@@ -70,6 +70,8 @@ export type ComboRelayOptions = {
|
||||
mode?: string | null;
|
||||
/** Per-request X-OmniRoute-Budget value (hard cost ceiling in USD) — #6023. */
|
||||
budgetCap?: number | null;
|
||||
/** Per-request X-OmniRoute-Budget-Fallback value ("cheapest" | "strict") — #3470. */
|
||||
budgetFallback?: "cheapest" | "strict" | null;
|
||||
[key: string]: unknown;
|
||||
};
|
||||
|
||||
|
||||
@@ -25,10 +25,7 @@ import { acceptHeaderForcesStream } from "@omniroute/open-sse/utils/aiSdkCompat.
|
||||
import { isSelfInflictedUpstreamTimeout } from "@omniroute/open-sse/handlers/chatCore/cooldownClassification.ts";
|
||||
import { applyNoThinkingAlias } from "@omniroute/open-sse/utils/noThinkingAlias.ts";
|
||||
import { handleComboChat } from "@omniroute/open-sse/services/combo.ts";
|
||||
import {
|
||||
resolveRequestModePack,
|
||||
parseRequestBudgetCap,
|
||||
} from "@omniroute/open-sse/services/autoCombo/requestControls.ts";
|
||||
import { resolveRequestAutoControls } from "@omniroute/open-sse/services/autoCombo/requestControls.ts";
|
||||
import { resolveComboConfig } from "@omniroute/open-sse/services/comboConfig.ts";
|
||||
import { injectHandoffIntoBody } from "@omniroute/open-sse/services/contextHandoff.ts";
|
||||
import {
|
||||
@@ -766,17 +763,13 @@ export async function handleChat(
|
||||
]);
|
||||
const relayConfig =
|
||||
combo.strategy === "context-relay" ? resolveComboConfig(combo, settings) : null;
|
||||
// Per-request Auto-Combo controls (#6023 / #6024 / #6025): steer an `auto`
|
||||
// combo on this single request without mutating its stored config.
|
||||
const requestModeHeader = request.headers.get("x-omniroute-mode")?.trim() || null;
|
||||
const requestBudgetHeader = request.headers.get("x-omniroute-budget")?.trim() || null;
|
||||
const perRequestMode = resolveRequestModePack(requestModeHeader);
|
||||
const perRequestBudgetCap = parseRequestBudgetCap(requestBudgetHeader);
|
||||
// Per-request Auto-Combo controls (#6023 / #6024 / #6025 / #3470): steer an
|
||||
// `auto` combo on this single request without mutating its stored config.
|
||||
const perRequestAutoControls = resolveRequestAutoControls(request.headers);
|
||||
const relayOptions =
|
||||
combo.strategy === "context-relay" ||
|
||||
bypassProviderQuotaPolicy ||
|
||||
perRequestMode.override ||
|
||||
perRequestBudgetCap !== undefined
|
||||
Object.keys(perRequestAutoControls).length > 0
|
||||
? {
|
||||
...(combo.strategy === "context-relay"
|
||||
? {
|
||||
@@ -785,8 +778,7 @@ export async function handleChat(
|
||||
}
|
||||
: {}),
|
||||
...(bypassProviderQuotaPolicy ? { bypassProviderQuotaPolicy: true } : {}),
|
||||
...(perRequestMode.override ? { mode: requestModeHeader } : {}),
|
||||
...(perRequestBudgetCap !== undefined ? { budgetCap: perRequestBudgetCap } : {}),
|
||||
...perRequestAutoControls,
|
||||
}
|
||||
: undefined;
|
||||
telemetry.endPhase();
|
||||
|
||||
161
tests/unit/auto-combo-budget-fallback-3470.test.ts
Normal file
161
tests/unit/auto-combo-budget-fallback-3470.test.ts
Normal file
@@ -0,0 +1,161 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import {
|
||||
BudgetExceededError,
|
||||
selectProvider,
|
||||
} from "../../open-sse/services/autoCombo/engine.ts";
|
||||
import {
|
||||
parseRequestBudgetFallback,
|
||||
resolveRequestAutoControls,
|
||||
} from "../../open-sse/services/autoCombo/requestControls.ts";
|
||||
import { getSelfHealingManager } from "../../open-sse/services/autoCombo/selfHealing.ts";
|
||||
import { DEFAULT_WEIGHTS } from "../../open-sse/services/autoCombo/scoring.ts";
|
||||
|
||||
// #3470 — Auto-combo transparency + budget controls: `budgetFallback: "strict"`
|
||||
// must refuse to select (instead of silently overspending) when EVERY candidate
|
||||
// exceeds the configured `budgetCap`.
|
||||
|
||||
const healer = getSelfHealingManager();
|
||||
const originalRandom = Math.random;
|
||||
|
||||
function resetHealer() {
|
||||
healer.exclusions.clear();
|
||||
healer.incidentMode = false;
|
||||
}
|
||||
|
||||
const baseConfig = {
|
||||
id: "auto-main",
|
||||
name: "Auto Main Budget Strict",
|
||||
type: "auto" as const,
|
||||
candidatePool: [],
|
||||
weights: DEFAULT_WEIGHTS,
|
||||
explorationRate: 0,
|
||||
};
|
||||
|
||||
const overBudgetCandidates = [
|
||||
{
|
||||
provider: "premium",
|
||||
model: "gpt-4o",
|
||||
quotaRemaining: 99,
|
||||
quotaTotal: 100,
|
||||
circuitBreakerState: "CLOSED",
|
||||
costPer1MTokens: 12000,
|
||||
p95LatencyMs: 100,
|
||||
latencyStdDev: 10,
|
||||
errorRate: 0.01,
|
||||
accountTier: "ultra",
|
||||
quotaResetIntervalSecs: 60,
|
||||
},
|
||||
{
|
||||
provider: "cheap",
|
||||
model: "gpt-4o-mini",
|
||||
quotaRemaining: 60,
|
||||
quotaTotal: 100,
|
||||
circuitBreakerState: "CLOSED",
|
||||
costPer1MTokens: 100,
|
||||
p95LatencyMs: 900,
|
||||
latencyStdDev: 50,
|
||||
errorRate: 0.02,
|
||||
accountTier: "free",
|
||||
quotaResetIntervalSecs: 86400,
|
||||
},
|
||||
];
|
||||
|
||||
test.beforeEach(() => {
|
||||
resetHealer();
|
||||
Math.random = originalRandom;
|
||||
});
|
||||
|
||||
test.afterEach(() => {
|
||||
resetHealer();
|
||||
Math.random = originalRandom;
|
||||
});
|
||||
|
||||
test("selectProvider throws BudgetExceededError when budgetFallback='strict' and every candidate exceeds budgetCap", () => {
|
||||
assert.throws(
|
||||
() =>
|
||||
selectProvider(
|
||||
{ ...baseConfig, budgetCap: 0.001, budgetFallback: "strict" },
|
||||
overBudgetCandidates,
|
||||
"default"
|
||||
),
|
||||
BudgetExceededError
|
||||
);
|
||||
});
|
||||
|
||||
test("BudgetExceededError message reports the cap and the cheapest candidate's cost, no stack leak", () => {
|
||||
try {
|
||||
selectProvider(
|
||||
{ ...baseConfig, budgetCap: 0.001, budgetFallback: "strict" },
|
||||
overBudgetCandidates,
|
||||
"default"
|
||||
);
|
||||
assert.fail("expected selectProvider to throw");
|
||||
} catch (err) {
|
||||
assert.ok(err instanceof BudgetExceededError);
|
||||
assert.match(err.message, /budget cap of \$0\.0010/);
|
||||
assert.ok(!err.message.includes("at /"));
|
||||
}
|
||||
});
|
||||
|
||||
test("selectProvider still falls back to cheapest when budgetFallback is 'cheapest' (default/legacy)", () => {
|
||||
const result = selectProvider(
|
||||
{ ...baseConfig, budgetCap: 0.001, budgetFallback: "cheapest" },
|
||||
overBudgetCandidates,
|
||||
"default"
|
||||
);
|
||||
assert.equal(result.provider, "cheap");
|
||||
});
|
||||
|
||||
test("selectProvider defaults to cheapest fallback when budgetFallback is unset (backward compatible)", () => {
|
||||
const result = selectProvider({ ...baseConfig, budgetCap: 0.001 }, overBudgetCandidates, "default");
|
||||
assert.equal(result.provider, "cheap");
|
||||
});
|
||||
|
||||
test("selectProvider with strict fallback still picks a within-budget candidate normally", () => {
|
||||
const result = selectProvider(
|
||||
{ ...baseConfig, budgetCap: 1, budgetFallback: "strict" },
|
||||
overBudgetCandidates,
|
||||
"default"
|
||||
);
|
||||
assert.equal(result.provider, "cheap");
|
||||
});
|
||||
|
||||
test("parseRequestBudgetFallback: accepts 'strict' and its aliases", () => {
|
||||
assert.equal(parseRequestBudgetFallback("strict"), "strict");
|
||||
assert.equal(parseRequestBudgetFallback("BLOCK"), "strict");
|
||||
assert.equal(parseRequestBudgetFallback(" hard "), "strict");
|
||||
});
|
||||
|
||||
test("parseRequestBudgetFallback: accepts 'cheapest' and its aliases", () => {
|
||||
assert.equal(parseRequestBudgetFallback("cheapest"), "cheapest");
|
||||
assert.equal(parseRequestBudgetFallback("Cheapest-Viable"), "cheapest");
|
||||
assert.equal(parseRequestBudgetFallback("soft"), "cheapest");
|
||||
});
|
||||
|
||||
test("parseRequestBudgetFallback: ignores unknown/empty/non-string values", () => {
|
||||
assert.equal(parseRequestBudgetFallback("garbage"), undefined);
|
||||
assert.equal(parseRequestBudgetFallback(""), undefined);
|
||||
assert.equal(parseRequestBudgetFallback(null), undefined);
|
||||
assert.equal(parseRequestBudgetFallback(42), undefined);
|
||||
});
|
||||
|
||||
test("resolveRequestAutoControls: aggregates mode/budget/budgetFallback headers, omitting unset ones", () => {
|
||||
const headers = new Headers({
|
||||
"x-omniroute-mode": "fast",
|
||||
"x-omniroute-budget": "0.05",
|
||||
"x-omniroute-budget-fallback": "strict",
|
||||
});
|
||||
const controls = resolveRequestAutoControls(headers);
|
||||
assert.deepEqual(controls, {
|
||||
mode: "fast",
|
||||
budgetCap: 0.05,
|
||||
budgetFallback: "strict",
|
||||
});
|
||||
});
|
||||
|
||||
test("resolveRequestAutoControls: returns an empty object when no auto-combo headers are present", () => {
|
||||
const controls = resolveRequestAutoControls(new Headers());
|
||||
assert.deepEqual(controls, {});
|
||||
});
|
||||
Reference in New Issue
Block a user