mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-19 05:32:19 +03:00
feat(sse): per-request Auto-Combo controls via X-OmniRoute-Mode / X-OmniRoute-Budget headers
Steer an `auto` combo on a single request without mutating its stored config: - X-OmniRoute-Mode (#6024/#6025): friendly presets (fast/balanced/quality/cheap/ reliable/offline) or a raw mode-pack name; balanced/default force default weights. - X-OmniRoute-Budget (#6023): hard per-request USD cost ceiling. Pure resolvers in open-sse/services/autoCombo/requestControls.ts feed the engine's existing config.modePack / config.budgetCap inputs (no engine changes). Threaded via relayOptions (chat.ts) into handleComboChat's auto path; unknown/garbage values are ignored so saved config is preserved. Docs: docs/routing/AUTO-COMBO.md. Closes #6023, #6024, #6025. Regression guard: tests/unit/auto-combo-request-controls-6024.test.ts (5).
This commit is contained in:
@@ -19,6 +19,7 @@
|
||||
- **feat(usage):** add on-demand period-scoped usage-data reset (Settings → System Storage) with a purge API and time-window selector.
|
||||
- **feat(claude-code):** add an opt-in auto-permission classifier compat mode (off/auto/always) for Claude Code, toggleable from the CLI Code settings.
|
||||
- **feat(providers):** add optional client-identity header profiles for compatible nodes — preset User-Agent/fingerprint headers (e.g. matching a known CLI) merged into the existing customHeaders field.
|
||||
- **feat(autoCombo):** add **per-request Auto-Combo controls** via two headers ([#6024](https://github.com/diegosouzapw/OmniRoute/issues/6024) / [#6025](https://github.com/diegosouzapw/OmniRoute/issues/6025) / [#6023](https://github.com/diegosouzapw/OmniRoute/issues/6023)) — `X-OmniRoute-Mode` steers an `auto` combo's scoring for a single request (friendly presets `fast`/`balanced`/`quality`/`cheap`/`reliable`/`offline` **or** a raw mode-pack name; `balanced` forces the default weights), and `X-OmniRoute-Budget` sets a hard per-request USD cost ceiling. Both override the combo's stored config only for the request that carries them; unknown/garbage values are ignored so the saved config is preserved. The resolvers are pure (`open-sse/services/autoCombo/requestControls.ts`) and feed the engine's existing `config.modePack` / `config.budgetCap` inputs — no engine changes. Regression guard: `tests/unit/auto-combo-request-controls-6024.test.ts` (5). (thanks @chirag127)
|
||||
|
||||
### 🔧 Bug Fixes
|
||||
|
||||
|
||||
@@ -148,6 +148,30 @@ Notes:
|
||||
- **quality-first** → taskFit 0.37 + stability 0.15 (best model for the task, consistent)
|
||||
- **offline-friendly** → quota 0.37 + health 0.28 (max headroom regardless of speed/cost)
|
||||
|
||||
### Per-Request Controls (headers) — #6023 / #6024 / #6025
|
||||
|
||||
An `auto` combo can be steered **per request** via two headers, without mutating the
|
||||
combo's stored config. These apply only to the `auto` strategy and only for the request
|
||||
that carries them; the combo's saved `modePack`/`budgetCap` are used when the header is
|
||||
absent.
|
||||
|
||||
| Header | Accepts | Effect |
|
||||
| :------------------- | :-------------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------------------------------------------------- |
|
||||
| `X-OmniRoute-Mode` | a preset alias (`fast`, `balanced`, `quality`, `cheap`, `reliable`, `offline`) or a raw pack name (`ship-fast`, `cost-saver`, `quality-first`, `offline-friendly`, `reliability-first`) | Overrides the scoring weights for this request. `balanced`/`default` force the default weights (no pack). Unknown values are ignored (config preserved). |
|
||||
| `X-OmniRoute-Budget` | a positive number (max USD per request) | Hard cost ceiling: candidates whose estimated cost exceeds it are filtered before selection, falling back to the cheapest healthy candidate if all exceed. Non-positive/garbage values are ignored. |
|
||||
|
||||
```bash
|
||||
# Force the fastest profile and cap this request at $0.05
|
||||
curl -sS http://localhost:20128/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "X-OmniRoute-Mode: fast" \
|
||||
-H "X-OmniRoute-Budget: 0.05" \
|
||||
-d '{"model":"auto","messages":[{"role":"user","content":"hi"}]}'
|
||||
```
|
||||
|
||||
Resolution is a pure function (`open-sse/services/autoCombo/requestControls.ts`); the
|
||||
resolved values feed the engine's existing `config.modePack` / `config.budgetCap` inputs.
|
||||
|
||||
## All Routing Strategies
|
||||
|
||||
OmniRoute's combo engine supports **17 routing strategies** (declared in `src/shared/constants/routingStrategies.ts` → `ROUTING_STRATEGY_VALUES`). The Auto Combo engine itself is exposed under the `auto` strategy; the others are available for persisted combos.
|
||||
|
||||
80
open-sse/services/autoCombo/requestControls.ts
Normal file
80
open-sse/services/autoCombo/requestControls.ts
Normal file
@@ -0,0 +1,80 @@
|
||||
/**
|
||||
* Per-request Auto-Combo routing controls (#6023 / #6024 / #6025).
|
||||
*
|
||||
* These let a caller steer an `auto` combo on a single request via response-safe
|
||||
* request headers, without changing the combo's stored config:
|
||||
*
|
||||
* X-OmniRoute-Mode: fast | balanced | quality | <raw mode-pack name> (#6024/#6025)
|
||||
* X-OmniRoute-Budget: <max USD per request> (#6023)
|
||||
*
|
||||
* Both resolvers are pure so they can be unit-tested and reused by the entry
|
||||
* handler (src/sse/handlers/chat.ts) and the combo router (open-sse/services/combo.ts).
|
||||
* The resolved values feed the auto-combo engine's existing `config.modePack` /
|
||||
* `config.budgetCap` inputs — no engine changes required.
|
||||
*/
|
||||
|
||||
import { MODE_PACKS } from "./modePacks";
|
||||
|
||||
/**
|
||||
* Friendly latency-vs-quality preset aliases (#6024). These map human-facing
|
||||
* preset names to the concrete scoring mode packs the engine already ships.
|
||||
* `balanced`/`default` are handled specially (they mean "no pack" = default weights).
|
||||
*/
|
||||
const MODE_PACK_ALIASES: Record<string, string> = {
|
||||
fast: "ship-fast",
|
||||
fastest: "ship-fast",
|
||||
speed: "ship-fast",
|
||||
quality: "quality-first",
|
||||
best: "quality-first",
|
||||
cheap: "cost-saver",
|
||||
cost: "cost-saver",
|
||||
saver: "cost-saver",
|
||||
reliable: "reliability-first",
|
||||
offline: "offline-friendly",
|
||||
};
|
||||
|
||||
export interface RequestModePack {
|
||||
/** True when the request explicitly selected a mode (overrides combo config). */
|
||||
override: boolean;
|
||||
/** Resolved mode-pack name, or undefined for the balanced/default profile. */
|
||||
modePack: string | undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the `X-OmniRoute-Mode` header value into a mode-pack override.
|
||||
*
|
||||
* - A friendly alias (`fast`, `quality`, `cheap`, …) or a raw mode-pack name
|
||||
* (`ship-fast`, `quality-first`, …) → `{ override: true, modePack: <name> }`.
|
||||
* - `balanced` / `default` → `{ override: true, modePack: undefined }` (default weights).
|
||||
* - Unknown / empty / non-string → `{ override: false }` so the combo's own
|
||||
* stored `modePack` config is preserved.
|
||||
*/
|
||||
export function resolveRequestModePack(input: unknown): RequestModePack {
|
||||
const noOverride: RequestModePack = { override: false, modePack: undefined };
|
||||
if (typeof input !== "string") return noOverride;
|
||||
const key = input.trim().toLowerCase();
|
||||
if (!key) return noOverride;
|
||||
if (key === "balanced" || key === "default") return { override: true, modePack: undefined };
|
||||
if (Object.prototype.hasOwnProperty.call(MODE_PACKS, key)) {
|
||||
return { override: true, modePack: key };
|
||||
}
|
||||
const alias = MODE_PACK_ALIASES[key];
|
||||
if (alias) return { override: true, modePack: alias };
|
||||
return noOverride;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse the `X-OmniRoute-Budget` header into a hard per-request cost ceiling (USD).
|
||||
* Only a finite, strictly-positive amount is accepted; anything else returns
|
||||
* `undefined` so the combo's own stored `budgetCap` (if any) stays in effect.
|
||||
*/
|
||||
export function parseRequestBudgetCap(input: unknown): number | undefined {
|
||||
const n =
|
||||
typeof input === "number"
|
||||
? input
|
||||
: typeof input === "string"
|
||||
? Number(input.trim())
|
||||
: NaN;
|
||||
if (!Number.isFinite(n) || n <= 0) return undefined;
|
||||
return n;
|
||||
}
|
||||
@@ -58,6 +58,10 @@ import { emit } from "../../src/lib/events/eventBus";
|
||||
import { notifyWebhookEvent } from "../../src/lib/webhookDispatcher";
|
||||
import { classifyWithConfig } from "./intentClassifier.ts";
|
||||
import { selectProvider as selectAutoProvider } from "./autoCombo/engine.ts";
|
||||
import {
|
||||
resolveRequestModePack,
|
||||
parseRequestBudgetCap,
|
||||
} from "./autoCombo/requestControls.ts";
|
||||
import { selectWithStrategy } from "./autoCombo/routerStrategy.ts";
|
||||
import { parseAutoPrefix } from "./autoCombo/autoPrefix.ts";
|
||||
import { handlePipelineCombo, buildPipelineResponse } from "./autoCombo/pipelineRouter.ts";
|
||||
@@ -1151,11 +1155,27 @@ export async function handleComboChat({
|
||||
const explorationRate = Number.isFinite(Number(autoConfigSource.explorationRate))
|
||||
? Number(autoConfigSource.explorationRate)
|
||||
: 0.05;
|
||||
const budgetCap = Number.isFinite(Number(autoConfigSource.budgetCap))
|
||||
// Per-request overrides (#6023 / #6024 / #6025): X-OmniRoute-Budget and
|
||||
// X-OmniRoute-Mode headers are threaded here via relayOptions and take
|
||||
// precedence over the combo's stored config for this single request.
|
||||
const configBudgetCap = Number.isFinite(Number(autoConfigSource.budgetCap))
|
||||
? Number(autoConfigSource.budgetCap)
|
||||
: undefined;
|
||||
const modePack =
|
||||
const requestBudgetCap = parseRequestBudgetCap(relayOptions?.budgetCap);
|
||||
const budgetCap = requestBudgetCap ?? configBudgetCap;
|
||||
|
||||
const configModePack =
|
||||
typeof autoConfigSource.modePack === "string" ? autoConfigSource.modePack : undefined;
|
||||
const requestModePack = resolveRequestModePack(relayOptions?.mode);
|
||||
const modePack = requestModePack.override ? requestModePack.modePack : configModePack;
|
||||
if (requestModePack.override || requestBudgetCap !== undefined) {
|
||||
log.debug?.(
|
||||
"COMBO",
|
||||
`Auto strategy: per-request controls applied (mode=${
|
||||
requestModePack.override ? (requestModePack.modePack ?? "balanced") : "—"
|
||||
}, budgetCap=${requestBudgetCap ?? "—"})`
|
||||
);
|
||||
}
|
||||
const resetWindowConfig = resolveResetWindowConfig(autoConfigSource);
|
||||
const slaPolicy = resolveSlaRoutingPolicy(autoConfigSource);
|
||||
|
||||
|
||||
@@ -64,6 +64,10 @@ export type ComboRelayOptions = {
|
||||
sessionId?: string | null;
|
||||
config?: Record<string, unknown> | null;
|
||||
bypassProviderQuotaPolicy?: boolean;
|
||||
/** Per-request X-OmniRoute-Mode value (auto-combo preset / mode-pack name) — #6024/#6025. */
|
||||
mode?: string | null;
|
||||
/** Per-request X-OmniRoute-Budget value (hard cost ceiling in USD) — #6023. */
|
||||
budgetCap?: number | null;
|
||||
[key: string]: unknown;
|
||||
};
|
||||
|
||||
|
||||
@@ -23,6 +23,10 @@ import { acceptHeaderForcesStream } from "@omniroute/open-sse/utils/aiSdkCompat.
|
||||
import { isSelfInflictedUpstreamTimeout } from "@omniroute/open-sse/handlers/chatCore/cooldownClassification.ts";
|
||||
import { applyNoThinkingAlias } from "@omniroute/open-sse/utils/noThinkingAlias.ts";
|
||||
import { handleComboChat } from "@omniroute/open-sse/services/combo.ts";
|
||||
import {
|
||||
resolveRequestModePack,
|
||||
parseRequestBudgetCap,
|
||||
} from "@omniroute/open-sse/services/autoCombo/requestControls.ts";
|
||||
import { resolveComboConfig } from "@omniroute/open-sse/services/comboConfig.ts";
|
||||
import { injectHandoffIntoBody } from "@omniroute/open-sse/services/contextHandoff.ts";
|
||||
import {
|
||||
@@ -645,8 +649,17 @@ export async function handleChat(
|
||||
]);
|
||||
const relayConfig =
|
||||
combo.strategy === "context-relay" ? resolveComboConfig(combo, settings) : null;
|
||||
// Per-request Auto-Combo controls (#6023 / #6024 / #6025): steer an `auto`
|
||||
// combo on this single request without mutating its stored config.
|
||||
const requestModeHeader = request.headers.get("x-omniroute-mode")?.trim() || null;
|
||||
const requestBudgetHeader = request.headers.get("x-omniroute-budget")?.trim() || null;
|
||||
const perRequestMode = resolveRequestModePack(requestModeHeader);
|
||||
const perRequestBudgetCap = parseRequestBudgetCap(requestBudgetHeader);
|
||||
const relayOptions =
|
||||
combo.strategy === "context-relay" || bypassProviderQuotaPolicy
|
||||
combo.strategy === "context-relay" ||
|
||||
bypassProviderQuotaPolicy ||
|
||||
perRequestMode.override ||
|
||||
perRequestBudgetCap !== undefined
|
||||
? {
|
||||
...(combo.strategy === "context-relay"
|
||||
? {
|
||||
@@ -655,6 +668,8 @@ export async function handleChat(
|
||||
}
|
||||
: {}),
|
||||
...(bypassProviderQuotaPolicy ? { bypassProviderQuotaPolicy: true } : {}),
|
||||
...(perRequestMode.override ? { mode: requestModeHeader } : {}),
|
||||
...(perRequestBudgetCap !== undefined ? { budgetCap: perRequestBudgetCap } : {}),
|
||||
}
|
||||
: undefined;
|
||||
telemetry.endPhase();
|
||||
|
||||
67
tests/unit/auto-combo-request-controls-6024.test.ts
Normal file
67
tests/unit/auto-combo-request-controls-6024.test.ts
Normal file
@@ -0,0 +1,67 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
// Regression guard for #6024 / #6025 / #6023 — per-request auto-combo routing controls.
|
||||
// A caller can steer an `auto` combo per request via two headers:
|
||||
// X-OmniRoute-Mode: fast | balanced | quality | <raw mode-pack name> (#6024/#6025)
|
||||
// X-OmniRoute-Budget: <max USD per request> (#6023)
|
||||
// The pure resolvers below turn the raw header values into an override the
|
||||
// auto-combo engine already knows how to consume (config.modePack / config.budgetCap).
|
||||
|
||||
const { resolveRequestModePack, parseRequestBudgetCap } = await import(
|
||||
"../../open-sse/services/autoCombo/requestControls.ts"
|
||||
);
|
||||
|
||||
test("#6024 friendly presets map to mode packs and override combo config", () => {
|
||||
assert.deepEqual(resolveRequestModePack("fast"), { override: true, modePack: "ship-fast" });
|
||||
assert.deepEqual(resolveRequestModePack("quality"), {
|
||||
override: true,
|
||||
modePack: "quality-first",
|
||||
});
|
||||
assert.deepEqual(resolveRequestModePack("cheap"), { override: true, modePack: "cost-saver" });
|
||||
assert.deepEqual(resolveRequestModePack("reliable"), {
|
||||
override: true,
|
||||
modePack: "reliability-first",
|
||||
});
|
||||
});
|
||||
|
||||
test("#6024 'balanced'/'default' override to the default profile (no pack)", () => {
|
||||
assert.deepEqual(resolveRequestModePack("balanced"), { override: true, modePack: undefined });
|
||||
assert.deepEqual(resolveRequestModePack("default"), { override: true, modePack: undefined });
|
||||
});
|
||||
|
||||
test("#6025 raw mode-pack names pass through (case-insensitive, trimmed)", () => {
|
||||
assert.deepEqual(resolveRequestModePack("ship-fast"), {
|
||||
override: true,
|
||||
modePack: "ship-fast",
|
||||
});
|
||||
assert.deepEqual(resolveRequestModePack(" Quality-First "), {
|
||||
override: true,
|
||||
modePack: "quality-first",
|
||||
});
|
||||
});
|
||||
|
||||
test("#6025 unknown/empty/non-string input does NOT override (keeps combo config)", () => {
|
||||
for (const bad of ["", " ", "not-a-real-pack", null, undefined, 42, {}]) {
|
||||
assert.deepEqual(
|
||||
resolveRequestModePack(bad as unknown),
|
||||
{ override: false, modePack: undefined },
|
||||
`input ${JSON.stringify(bad)} must not override`
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test("#6023 budget header parses a positive USD amount, rejects garbage", () => {
|
||||
assert.equal(parseRequestBudgetCap("0.05"), 0.05);
|
||||
assert.equal(parseRequestBudgetCap("2"), 2);
|
||||
assert.equal(parseRequestBudgetCap(1.5), 1.5);
|
||||
assert.equal(parseRequestBudgetCap(" 0.5 "), 0.5);
|
||||
// rejected → undefined (fall back to combo config)
|
||||
for (const bad of ["0", "-1", "abc", "", " ", null, undefined, NaN, Infinity, 0, -3]) {
|
||||
assert.equal(
|
||||
parseRequestBudgetCap(bad as unknown),
|
||||
undefined,
|
||||
`input ${JSON.stringify(bad)} must be rejected`
|
||||
);
|
||||
}
|
||||
});
|
||||
Reference in New Issue
Block a user