feat(settings): 9router-style Routing Strategy card + sticky parity (#6678)

* feat(dashboard): 9router-parity Routing Strategy card + provider/combo sticky override (#6678)

Add a Routing Strategy settings card (Settings -> Routing) surfacing account
round-robin/sticky-limit knobs plus a new combo-level sticky round-robin
(comboStickyRoundRobinLimit), and a per-provider account-routing override
(providerStrategies) wired into getProviderCredentials() ahead of the global
fallback strategy. Rebased onto release/v3.8.47 (credit-preserving
reconstruction: unrelated package.json/electron/proxyDispatcher drift from the
PR's stale base was dropped, only the author's own 12 files were re-applied).
Split ProviderAccountRoutingCard/RoutingStrategyCard into smaller
hook+subcomponent pieces to stay under the frozen complexity/file-size gates;
rebaselined ProviderDetailPageClient.tsx/auth.ts's frozen file-size caps for
the small additive growth.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* chore(quality): register combo-rr-sticky-9router.test.ts in stryker tap.testFiles (#6678)

CI's Fast Quality Gates -> check:mutation-test-coverage --strict flagged the new
test as missing from stryker.conf.json's tap.testFiles (it covers the mutated
module open-sse/services/combo/rrState.ts). Adds the single entry, alphabetized
next to the existing combo-rr-fallback-advance-948.test.ts.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* fix(changelog): restore CHANGELOG bullets eaten by release sync

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* fix(changelog): re-restore CHANGELOG bullet after further release sync

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* fix(changelog): re-restore CHANGELOG bullet after further release sync

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@gmail.com>
Co-authored-by: SeaXen <SeaXen@users.noreply.github.com>
This commit is contained in:
SeaXen
2026-07-10 03:22:43 +06:00
committed by GitHub
parent d76aa40ee6
commit 6557f44bd2
15 changed files with 506 additions and 12 deletions

View File

@@ -12,6 +12,7 @@ _Living section — bullets land here as PRs merge into `release/v3.8.47` (paral
- **Provider/model param filters**: config-driven parameter denylist/allowlist per provider/model with auto-learn from upstream 400s (#6649 — thanks @ThongAccount, closes #6625)
- **Per-combo reasoning token buffer toggle**: the combo builder now exposes an explicit checkbox for the `#3587` reasoning-model `max_tokens` buffer, defaulting to the existing enabled behavior, so a combo can opt out without hand-editing raw JSON config (#6702 — thanks @xz-dev)
- **feat(dashboard):** 9router-parity **Routing Strategy** settings card on Settings → Routing, plus a per-provider account-routing override on the provider detail page ([#6678](https://github.com/diegosouzapw/OmniRoute/pull/6678)) — surfaces the existing account round-robin / sticky-limit knobs and adds a new combo-level sticky round-robin (`comboStickyRoundRobinLimit`, resolved via `resolveComboStickyRoundRobinLimit()` — per-combo → global combo sticky → account sticky cascade) so combo targets can batch calls per target the same way account fallback already does. A new `providerStrategies` setting (Zod-validated map, `src/shared/validation/settingsSchemas.ts`) lets a specific provider override the global `fallbackStrategy`/`stickyRoundRobinLimit` without touching the account-wide default, wired into `getProviderCredentials()` (`src/sse/services/auth.ts`) ahead of the global fallback. Regression guard: `tests/unit/combo-rr-sticky-9router.test.ts`, `tests/unit/settings-ui-layout-static.test.ts`. (thanks @SeaXen)
### 🐛 Bug Fixes

View File

@@ -202,7 +202,7 @@
"src/app/(dashboard)/dashboard/endpoint/EndpointPageClient.tsx": 2612,
"src/app/(dashboard)/dashboard/health/page.tsx": 1091,
"src/app/(dashboard)/dashboard/playground/components/tabs/ApiTab.tsx": 847,
"src/app/(dashboard)/dashboard/providers/[id]/ProviderDetailPageClient.tsx": 784,
"src/app/(dashboard)/dashboard/providers/[id]/ProviderDetailPageClient.tsx": 786,
"src/app/(dashboard)/dashboard/providers/[id]/components/ConnectionRow.tsx": 942,
"src/app/(dashboard)/dashboard/providers/[id]/components/modals/AddApiKeyModal.tsx": 961,
"src/app/(dashboard)/dashboard/providers/[id]/components/modals/EditConnectionModal.tsx": 1278,
@@ -259,9 +259,10 @@
"src/shared/services/cliRuntime.ts": 1110,
"src/shared/validation/schemas.ts": 2523,
"_rebaseline_2026_06_28_5275_correlation_id_extract": "Extraction of the safe CorrelationId subset of #5275 (hartmark) — request correlation id stored in call_logs (migration 109) and returned via the X-Correlation-Id response header, WITHOUT the combo/resilience or build/lazy-loading changes (those stay in #5275). Own growth: callLogs.ts 975->985 (correlation_id column on CallLogSummaryRow + read/map), usageHistory.ts 983->988 (correlationId metadata normalize), chat.ts 1575->1632 (withCorrelationId response wiring + combo-failure log carrying correlationId), chatHelpers.ts new 811 (withCorrelationId helper + reqId threading; was 791<cap pre-feature). Cohesive request/logging chokepoint wiring; structural shrink of chat.ts tracked in #3501.",
"_rebaseline_2026_07_09_6678_routing_strategy_9router": "#6678 (SeaXen) — 9router-parity Routing Strategy settings card + per-provider/combo sticky-round-robin override. Own growth: ProviderDetailPageClient.tsx 784->786 (single ProviderAccountRoutingCard mount + import), auth.ts 2448->2458 (providerStrategies override resolution: fallbackStrategy/stickyRoundRobinLimit per-provider cascade in getProviderCredentials). Both additive, zero unrelated refactor; new UI/logic lives in new files (ProviderAccountRoutingCard.tsx, RoutingStrategyCard.tsx, rrState.ts::resolveComboStickyRoundRobinLimit). chat.ts value below reflects the current release tip (grown by other concurrent PRs, e.g. #6640), not this PR own change.",
"src/sse/handlers/chat.ts": 1796,
"src/sse/handlers/chatHelpers.ts": 876,
"src/sse/services/auth.ts": 2448,
"src/sse/services/auth.ts": 2458,
"open-sse/executors/default.ts": 877,
"open-sse/translator/request/openai-responses.ts": 902,
"open-sse/executors/kiro.ts": 944,

View File

@@ -112,6 +112,7 @@ import {
recordStickyRoundRobinSuccess,
getStickyWeightedExecutionKey,
recordStickyWeightedSuccess,
resolveComboStickyRoundRobinLimit,
} from "./combo/rrState.ts";
import {
validateResponseQuality,
@@ -898,10 +899,9 @@ export async function handleComboChat({
let runtimeStickyTargets: ResolvedComboUnit[] = runtimeUnits;
if (strategy === "round-robin") {
const perComboStickyLimit = (config as Record<string, unknown>).stickyRoundRobinLimit;
runtimeStickyLimit = clampStickyRoundRobinTargetLimit(
perComboStickyLimit !== undefined && perComboStickyLimit !== null
? perComboStickyLimit
: (settings as Record<string, unknown> | null)?.stickyRoundRobinLimit
runtimeStickyLimit = resolveComboStickyRoundRobinLimit(
perComboStickyLimit,
settings as Record<string, unknown> | null
);
const { startIndex, counter } = getStickyRoundRobinStartIndex(
combo.name,
@@ -2495,10 +2495,9 @@ async function handleRoundRobinCombo({
// sticky batching for both account fallback and combo targets. Values <= 1 preserve
// the historical one-request-per-target rotation.
const perComboStickyLimit = (config as Record<string, unknown>).stickyRoundRobinLimit;
const stickyLimit = clampStickyRoundRobinTargetLimit(
perComboStickyLimit !== undefined && perComboStickyLimit !== null
? perComboStickyLimit
: (settings as Record<string, unknown> | null)?.stickyRoundRobinLimit
const stickyLimit = resolveComboStickyRoundRobinLimit(
perComboStickyLimit,
settings as Record<string, unknown> | null
);
const stickyRoundRobinEnabled = stickyLimit > 1;
// Exhaustion-aware sticky: if the currently sticky target is no longer

View File

@@ -96,3 +96,22 @@ export function recordStickyWeightedSuccess(
weightedStickyTargets.set(comboName, { executionKey, successCount });
}
/**
* Sticky batch size for round-robin combo targets (9router parity).
* Per-combo config → comboStickyRoundRobinLimit → stickyRoundRobinLimit.
* Uses clampStickyRoundRobinTargetLimit defined above in this module.
*/
export function resolveComboStickyRoundRobinLimit(
perComboLimit: unknown,
settings: Record<string, unknown> | null | undefined
): number {
if (perComboLimit !== undefined && perComboLimit !== null) {
return clampStickyRoundRobinTargetLimit(perComboLimit);
}
const comboSticky = settings?.comboStickyRoundRobinLimit;
if (comboSticky !== undefined && comboSticky !== null) {
return clampStickyRoundRobinTargetLimit(comboSticky);
}
return clampStickyRoundRobinTargetLimit(settings?.stickyRoundRobinLimit);
}

View File

@@ -45,6 +45,7 @@ import CustomModelsSection from "./components/CustomModelsSection";
import ConnectionsListPanel from "./components/ConnectionsListPanel";
import CoolingConnectionsPanel from "./components/CoolingConnectionsPanel";
import ConnectionsHeaderToolbar from "./components/ConnectionsHeaderToolbar";
import ProviderAccountRoutingCard from "../../settings/components/ProviderAccountRoutingCard";
import ZedImportCard from "./components/ZedImportCard";
import ProviderPageHeader from "./components/ProviderPageHeader";
import CompatibleNodeCard from "./components/CompatibleNodeCard";
@@ -488,6 +489,7 @@ export default function ProviderDetailPageClient() {
)}
{!isUpstreamProxyProvider && !isFreeNoAuth && (
<Card>
<ProviderAccountRoutingCard providerKey={providerId} connectionCount={connections.length} />
<ConnectionsHeaderToolbar
providerId={providerId}
providerInfo={providerInfo}

View File

@@ -0,0 +1,132 @@
"use client";
import { useCallback, useEffect, useState } from "react";
import { Input, Select } from "@/shared/components";
import { useTranslations } from "next-intl";
import { ACCOUNT_FALLBACK_STRATEGY_VALUES } from "@/shared/constants/routingStrategies";
type ProviderStrategyOverride = {
fallbackStrategy?: string;
stickyRoundRobinLimit?: number;
};
type Props = {
providerKey: string;
connectionCount: number;
};
const STRATEGY_OPTIONS = ACCOUNT_FALLBACK_STRATEGY_VALUES.filter((v) =>
["fill-first", "round-robin", "priority", "p2c", "random", "least-used"].includes(v)
);
function clampProviderStickyLimit(raw: string): number {
const val = parseInt(raw, 10);
return Math.min(10, Math.max(1, Number.isNaN(val) ? 3 : val));
}
/** Loads/saves the per-provider account-routing override. Extracted out of the
* component body to keep ProviderAccountRoutingCard's own render function small. */
function useProviderAccountRoutingState(providerKey: string) {
const [strategy, setStrategy] = useState<string>("");
const [stickyLimit, setStickyLimit] = useState("3");
const [busy, setBusy] = useState(false);
const load = useCallback(async () => {
const res = await fetch("/api/settings", { cache: "no-store" });
if (!res.ok) return;
const data = await res.json();
const override = ((data?.providerStrategies || {}) as Record<string, ProviderStrategyOverride>)[
providerKey
];
setStrategy(override?.fallbackStrategy || "");
setStickyLimit(
override?.stickyRoundRobinLimit != null ? String(override.stickyRoundRobinLimit) : ""
);
}, [providerKey]);
useEffect(() => {
load().catch(console.error);
}, [load]);
const save = useCallback(
async (nextStrategy: string, nextSticky: string) => {
setBusy(true);
try {
const res = await fetch("/api/settings", { cache: "no-store" });
if (!res.ok) throw new Error("Failed to fetch current settings");
const data = await res.json();
const current = (data?.providerStrategies || {}) as Record<
string,
ProviderStrategyOverride
>;
const override: ProviderStrategyOverride = {};
if (nextStrategy) override.fallbackStrategy = nextStrategy;
if (nextStrategy === "round-robin" && nextSticky !== "") {
override.stickyRoundRobinLimit = clampProviderStickyLimit(nextSticky);
}
const updated = { ...current };
if (Object.keys(override).length === 0) delete updated[providerKey];
else updated[providerKey] = override;
const patchRes = await fetch("/api/settings", {
method: "PATCH",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ providerStrategies: updated }),
});
if (!patchRes.ok) throw new Error("Failed to save provider routing settings");
} catch (e) {
console.error(e);
} finally {
setBusy(false);
}
},
[providerKey]
);
return { strategy, setStrategy, stickyLimit, setStickyLimit, busy, save };
}
export default function ProviderAccountRoutingCard({ providerKey, connectionCount }: Props) {
const t = useTranslations("settings");
const { strategy, setStrategy, stickyLimit, setStickyLimit, busy, save } =
useProviderAccountRoutingState(providerKey);
if (connectionCount < 2) return null;
return (
<div className="mb-4 rounded-lg border border-border/60 bg-black/[0.02] dark:bg-white/[0.02] p-3">
<p className="text-sm font-medium">{t("providerAccountRoutingTitle")}</p>
<p className="text-xs text-text-muted mb-3">{t("providerAccountRoutingDesc")}</p>
<div className="grid grid-cols-1 sm:grid-cols-2 gap-3">
<Select
label={t("providerRoutingStrategy")}
disabled={busy}
value={strategy}
onChange={(e) => {
const v = e.target.value;
setStrategy(v);
save(v, stickyLimit).catch(console.error);
}}
>
<option value="">{t("providerRoutingInheritGlobal")}</option>
{STRATEGY_OPTIONS.map((opt) => (
<option key={opt} value={opt}>
{opt}
</option>
))}
</Select>
{strategy === "round-robin" && (
<Input
label={t("stickyLimit")}
type="number"
min={1}
max={10}
disabled={busy}
value={stickyLimit || "3"}
onChange={(e) => setStickyLimit(e.target.value)}
onBlur={() => save(strategy, stickyLimit).catch(console.error)}
/>
)}
</div>
</div>
);
}

View File

@@ -0,0 +1,282 @@
"use client";
import { useCallback, useEffect, useState } from "react";
import { Card, Input, Toggle } from "@/shared/components";
import { useTranslations } from "next-intl";
type RoutingSettings = {
fallbackStrategy?: string;
stickyRoundRobinLimit?: number | string;
comboStrategy?: string;
comboStickyRoundRobinLimit?: number | string;
};
type Translate = ReturnType<typeof useTranslations>;
async function patchSettings(body: Record<string, unknown>) {
const res = await fetch("/api/settings", {
method: "PATCH",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(body),
});
if (!res.ok) throw new Error("settings patch failed");
return res.json();
}
async function patchComboDefaultsStrategy(strategy: string) {
const res = await fetch("/api/settings/combo-defaults", {
method: "PATCH",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ comboDefaults: { strategy } }),
});
if (!res.ok) throw new Error("combo-defaults patch failed");
}
function numericLimit(value: number | string | undefined, fallback: number): number {
const n = typeof value === "number" ? value : parseInt(String(value), 10);
return Number.isFinite(n) ? n : fallback;
}
/** Loads/persists the routing-strategy settings and tracks loading/busy state.
* Extracted out of the component body to keep RoutingStrategyCard's own render
* function under the complexity/size gate. */
function useRoutingStrategySettings() {
const [settings, setSettings] = useState<RoutingSettings>({
fallbackStrategy: "fill-first",
stickyRoundRobinLimit: 3,
comboStrategy: "fallback",
comboStickyRoundRobinLimit: 1,
});
const [loading, setLoading] = useState(true);
const [busy, setBusy] = useState(false);
const load = useCallback(async () => {
setLoading(true);
try {
const res = await fetch("/api/settings", { cache: "no-store" });
if (!res.ok) return;
const data = await res.json();
setSettings({
fallbackStrategy: data?.fallbackStrategy || "fill-first",
stickyRoundRobinLimit: data?.stickyRoundRobinLimit ?? 3,
comboStrategy: data?.comboStrategy || "fallback",
comboStickyRoundRobinLimit: data?.comboStickyRoundRobinLimit ?? 1,
});
} finally {
setLoading(false);
}
}, []);
useEffect(() => {
load().catch(console.error);
}, [load]);
const run = useCallback(async (fn: () => Promise<void>) => {
setBusy(true);
try {
await fn();
} catch (e) {
console.error(e);
} finally {
setBusy(false);
}
}, []);
return { settings, setSettings, loading, busy, run };
}
type SectionProps = {
t: Translate;
busy: boolean;
settings: RoutingSettings;
setSettings: React.Dispatch<React.SetStateAction<RoutingSettings>>;
run: (fn: () => Promise<void>) => Promise<void>;
};
function AccountRoundRobinSection({ t, busy, settings, setSettings, run }: SectionProps) {
const accountRoundRobin = settings.fallbackStrategy === "round-robin";
return (
<>
<div className="flex items-start sm:items-center justify-between gap-4">
<div className="flex-1 min-w-0">
<p className="font-medium text-sm sm:text-base">{t("accountRoundRobin")}</p>
<p className="text-xs sm:text-sm text-text-muted">{t("accountRoundRobinDesc")}</p>
</div>
<Toggle
checked={accountRoundRobin}
disabled={busy}
onChange={() =>
run(async () => {
const next = accountRoundRobin ? "fill-first" : "round-robin";
const updated = await patchSettings({ fallbackStrategy: next });
setSettings((prev) => ({ ...prev, fallbackStrategy: updated.fallbackStrategy || next }));
})
}
/>
</div>
{accountRoundRobin && (
<div className="flex items-start sm:items-center justify-between gap-4 pt-2 border-t border-border/50">
<div className="flex-1 min-w-0">
<p className="font-medium text-sm sm:text-base">{t("stickyLimit")}</p>
<p className="text-xs sm:text-sm text-text-muted">{t("stickyLimitDesc")}</p>
</div>
<Input
type="number"
min={1}
max={10}
disabled={busy}
className="w-16 sm:w-20 text-center shrink-0"
value={settings.stickyRoundRobinLimit ?? 3}
onChange={(e) => setSettings((prev) => ({ ...prev, stickyRoundRobinLimit: e.target.value }))}
onBlur={() =>
run(async () => {
const limit = Math.min(
10,
Math.max(1, parseInt(String(settings.stickyRoundRobinLimit), 10) || 3)
);
const updated = await patchSettings({ stickyRoundRobinLimit: limit });
setSettings((prev) => ({
...prev,
stickyRoundRobinLimit: updated.stickyRoundRobinLimit ?? limit,
}));
})
}
/>
</div>
)}
</>
);
}
function ComboRoundRobinSection({ t, busy, settings, setSettings, run }: SectionProps) {
const comboRoundRobin = settings.comboStrategy === "round-robin";
return (
<>
<div className="flex items-start sm:items-center justify-between gap-4 pt-4 border-t border-border/50">
<div className="flex-1 min-w-0">
<p className="font-medium text-sm sm:text-base">{t("comboRoundRobin")}</p>
<p className="text-xs sm:text-sm text-text-muted">{t("comboRoundRobinDesc")}</p>
</div>
<Toggle
checked={comboRoundRobin}
disabled={busy}
onChange={() =>
run(async () => {
const next = comboRoundRobin ? "fallback" : "round-robin";
const defaultComboStrategy = next === "round-robin" ? "round-robin" : "priority";
await patchComboDefaultsStrategy(defaultComboStrategy);
const updated = await patchSettings({ comboStrategy: next });
setSettings((prev) => ({ ...prev, comboStrategy: updated.comboStrategy || next }));
})
}
/>
</div>
{comboRoundRobin && (
<div className="flex items-center justify-between pt-2 border-t border-border/50">
<div>
<p className="font-medium">{t("comboStickyLimit")}</p>
<p className="text-sm text-text-muted">{t("comboStickyLimitDesc")}</p>
</div>
<Input
type="number"
min={1}
max={100}
disabled={busy}
className="w-20 text-center"
value={settings.comboStickyRoundRobinLimit ?? 1}
onChange={(e) =>
setSettings((prev) => ({ ...prev, comboStickyRoundRobinLimit: e.target.value }))
}
onBlur={() =>
run(async () => {
const limit = Math.min(
100,
Math.max(1, parseInt(String(settings.comboStickyRoundRobinLimit), 10) || 1)
);
const updated = await patchSettings({ comboStickyRoundRobinLimit: limit });
setSettings((prev) => ({
...prev,
comboStickyRoundRobinLimit: updated.comboStickyRoundRobinLimit ?? limit,
}));
})
}
/>
</div>
)}
</>
);
}
function RoutingSummaryFooter({
t,
accountRoundRobin,
comboRoundRobin,
accountStickyDisplay,
comboStickyDisplay,
}: {
t: Translate;
accountRoundRobin: boolean;
comboRoundRobin: boolean;
accountStickyDisplay: number;
comboStickyDisplay: number;
}) {
return (
<p className="text-xs text-text-muted italic pt-2 border-t border-border/50">
{accountRoundRobin
? t("routingStrategyAccountSummary", { limit: accountStickyDisplay })
: t("routingStrategyFillFirstSummary")}
{comboRoundRobin
? t("routingStrategyComboSummary", { limit: comboStickyDisplay })
: t("routingStrategyComboFallbackSummary")}
</p>
);
}
/**
* 9router-style Routing Strategy card (Profile → Routing Strategy parity).
* Settings → Routing, first card on the page.
*/
export default function RoutingStrategyCard() {
const t = useTranslations("settings");
const tc = useTranslations("common");
const { settings, setSettings, loading, busy, run } = useRoutingStrategySettings();
const accountRoundRobin = settings.fallbackStrategy === "round-robin";
const comboRoundRobin = settings.comboStrategy === "round-robin";
const accountStickyDisplay = numericLimit(settings.stickyRoundRobinLimit, 3);
const comboStickyDisplay = numericLimit(settings.comboStickyRoundRobinLimit, 1);
return (
<Card>
<div className="flex items-center gap-3 mb-4">
<div className="p-2 rounded-lg bg-blue-500/10 text-blue-500 shrink-0">
<span className="material-symbols-outlined text-[20px]" aria-hidden="true">
route
</span>
</div>
<div>
<h3 className="text-base sm:text-lg font-semibold">{t("routingStrategyTitle")}</h3>
<p className="text-xs text-text-muted">{t("routingStrategySubtitle")}</p>
</div>
</div>
{loading ? (
<p className="text-sm text-text-muted">{tc("loading")}</p>
) : (
<div className="flex flex-col gap-4">
<AccountRoundRobinSection t={t} busy={busy} settings={settings} setSettings={setSettings} run={run} />
<ComboRoundRobinSection t={t} busy={busy} settings={settings} setSettings={setSettings} run={run} />
<RoutingSummaryFooter
t={t}
accountRoundRobin={accountRoundRobin}
comboRoundRobin={comboRoundRobin}
accountStickyDisplay={accountStickyDisplay}
comboStickyDisplay={comboStickyDisplay}
/>
</div>
)}
</Card>
);
}

View File

@@ -1,6 +1,7 @@
"use client";
import { useTranslations } from "next-intl";
import RoutingStrategyCard from "../components/RoutingStrategyCard";
import RoutingTab from "../components/RoutingTab";
import ModelRoutingSection from "@/shared/components/ModelRoutingSection";
import ComboDefaultsTab from "../components/ComboDefaultsTab";
@@ -13,6 +14,7 @@ export default function SettingsRoutingPage() {
return (
<div className="space-y-6">
<p className="text-sm text-text-muted">{t("routingSettingsIntro")}</p>
<RoutingStrategyCard />
<ComboDefaultsTab />
<ModelAliasesUnified />
<FallbackChainsEditor />

View File

@@ -5326,6 +5326,22 @@
"strictRandomDesc": "Shuffle deck — uses each account once before reshuffling",
"stickyLimit": "Sticky Limit",
"stickyLimitDesc": "Calls per account before switching when global account fallback uses Round Robin",
"routingStrategyTitle": "Routing Strategy",
"routingStrategySubtitle": "Match 9router: account round-robin, sticky limits, and combo rotation",
"accountRoundRobin": "Round Robin",
"accountRoundRobinDesc": "Cycle through accounts to distribute load",
"comboRoundRobin": "Combo Round Robin",
"comboRoundRobinDesc": "Cycle through combo targets instead of always starting with the first",
"comboStickyLimit": "Combo Sticky Limit",
"comboStickyLimitDesc": "Calls per combo target before switching",
"routingStrategyAccountSummary": "Distributing requests across accounts with {limit} calls per account.",
"routingStrategyFillFirstSummary": "Using accounts in priority order (Fill First).",
"routingStrategyComboSummary": " Combos rotate after {limit} call(s) per target.",
"routingStrategyComboFallbackSummary": " Combos use each combo's configured strategy (default priority/fallback).",
"providerAccountRoutingTitle": "Multi-account routing",
"providerAccountRoutingDesc": "Override global account strategy for this provider (9router parity).",
"providerRoutingStrategy": "Account strategy",
"providerRoutingInheritGlobal": "Inherit global default",
"modelAliases": "Model Aliases",
"modelAliasesTitle": "Model Aliases",
"modelAliasesDesc": "Remap model names using exact matches or wildcard patterns.",

View File

@@ -97,6 +97,9 @@ export async function getSettings() {
tailscaleUrl: "",
stickyRoundRobinLimit: 3,
disableSessionStickiness: false,
comboStrategy: "fallback",
comboStickyRoundRobinLimit: 1,
providerStrategies: {},
requestRetry: 3,
maxRetryIntervalSec: 30,
antigravitySignatureCacheMode: "enabled",

View File

@@ -183,6 +183,18 @@ export const updateSettingsSchema = z.object({
fallbackStrategy: z.enum(ACCOUNT_FALLBACK_STRATEGY_VALUES).optional(),
wildcardAliases: z.array(z.object({ pattern: z.string(), target: z.string() })).optional(),
stickyRoundRobinLimit: z.number().int().min(0).max(1000).optional(),
/** 9router parity: global combo expansion strategy (fallback vs round-robin). */
comboStrategy: z.enum(["fallback", "round-robin"]).optional(),
comboStickyRoundRobinLimit: z.number().int().min(1).max(100).optional(),
providerStrategies: z
.record(
z.string().trim().min(1),
z.object({
fallbackStrategy: z.enum(ACCOUNT_FALLBACK_STRATEGY_VALUES).optional(),
stickyRoundRobinLimit: z.number().int().min(1).max(1000).optional(),
})
)
.optional(),
// #6168: global session-stickiness opt-out (per-combo config overrides this).
disableSessionStickiness: z.boolean().optional(),
requestRetry: z.number().int().min(0).max(10).optional(),

View File

@@ -1477,7 +1477,13 @@ export async function getProviderCredentials(
const orderedConnections = withQuota;
const strategy = settings.fallbackStrategy || "fill-first";
const providerStrategyOverrides = (settings.providerStrategies || {}) as Record<
string,
{ fallbackStrategy?: string; stickyRoundRobinLimit?: number }
>;
const providerOverride = providerStrategyOverrides[resolvedId] || {};
const strategy =
providerOverride.fallbackStrategy || settings.fallbackStrategy || "fill-first";
let connection;
const affinityConnection = await selectSessionAffinityConnection(
@@ -1498,7 +1504,11 @@ export async function getProviderCredentials(
if (connection) {
// Session affinity selected a connection before global sticky routing.
} else if (strategy === "round-robin") {
const stickyLimit = toNumber((settings as Record<string, unknown>).stickyRoundRobinLimit, 3);
const stickyLimit = toNumber(
providerOverride.stickyRoundRobinLimit ??
(settings as Record<string, unknown>).stickyRoundRobinLimit,
3
);
// If excluding account(s) (fallback scenario), skip sticky logic and go straight to LRU.
// This prevents same-model retries from getting stuck on a failed account.

View File

@@ -139,6 +139,7 @@
"tests/unit/combo-roundrobin-compat-fallback-6238.test.ts",
"tests/unit/combo-routing-engine.test.ts",
"tests/unit/combo-rr-fallback-advance-948.test.ts",
"tests/unit/combo-rr-sticky-9router.test.ts",
"tests/unit/combo-scoring-inspector.test.ts",
"tests/unit/combo-selected-connection-success.test.ts",
"tests/unit/combo-sessionless-pin-3825.test.ts",

View File

@@ -0,0 +1,13 @@
import test from "node:test";
import assert from "node:assert/strict";
import { resolveComboStickyRoundRobinLimit } from "../../open-sse/services/combo/rrState.ts";
test("resolveComboStickyRoundRobinLimit prefers per-combo, then comboSticky, then account sticky", () => {
const settings = { stickyRoundRobinLimit: 3, comboStickyRoundRobinLimit: 2 };
assert.equal(resolveComboStickyRoundRobinLimit(5, settings), 5);
assert.equal(resolveComboStickyRoundRobinLimit(undefined, settings), 2);
assert.equal(
resolveComboStickyRoundRobinLimit(undefined, { stickyRoundRobinLimit: 7 }),
7
);
});

View File

@@ -99,6 +99,7 @@ test("Global Routing page renders top-level modules in the requested order", ()
const routingTab = readSrc("src/app/(dashboard)/dashboard/settings/components/RoutingTab.tsx");
assertInOrder(page, [
"<RoutingStrategyCard",
"<ComboDefaultsTab",
"<ModelAliasesUnified",
"<FallbackChainsEditor",