mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-03 13:52:09 +03:00
feat(settings): 9router-style Routing Strategy card + sticky parity (#6678)
* feat(dashboard): 9router-parity Routing Strategy card + provider/combo sticky override (#6678) Add a Routing Strategy settings card (Settings -> Routing) surfacing account round-robin/sticky-limit knobs plus a new combo-level sticky round-robin (comboStickyRoundRobinLimit), and a per-provider account-routing override (providerStrategies) wired into getProviderCredentials() ahead of the global fallback strategy. Rebased onto release/v3.8.47 (credit-preserving reconstruction: unrelated package.json/electron/proxyDispatcher drift from the PR's stale base was dropped, only the author's own 12 files were re-applied). Split ProviderAccountRoutingCard/RoutingStrategyCard into smaller hook+subcomponent pieces to stay under the frozen complexity/file-size gates; rebaselined ProviderDetailPageClient.tsx/auth.ts's frozen file-size caps for the small additive growth. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> * chore(quality): register combo-rr-sticky-9router.test.ts in stryker tap.testFiles (#6678) CI's Fast Quality Gates -> check:mutation-test-coverage --strict flagged the new test as missing from stryker.conf.json's tap.testFiles (it covers the mutated module open-sse/services/combo/rrState.ts). Adds the single entry, alphabetized next to the existing combo-rr-fallback-advance-948.test.ts. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> * fix(changelog): restore CHANGELOG bullets eaten by release sync Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> * fix(changelog): re-restore CHANGELOG bullet after further release sync Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> * fix(changelog): re-restore CHANGELOG bullet after further release sync Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> --------- Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@gmail.com> Co-authored-by: SeaXen <SeaXen@users.noreply.github.com>
This commit is contained in:
@@ -12,6 +12,7 @@ _Living section — bullets land here as PRs merge into `release/v3.8.47` (paral
|
||||
|
||||
- **Provider/model param filters**: config-driven parameter denylist/allowlist per provider/model with auto-learn from upstream 400s (#6649 — thanks @ThongAccount, closes #6625)
|
||||
- **Per-combo reasoning token buffer toggle**: the combo builder now exposes an explicit checkbox for the `#3587` reasoning-model `max_tokens` buffer, defaulting to the existing enabled behavior, so a combo can opt out without hand-editing raw JSON config (#6702 — thanks @xz-dev)
|
||||
- **feat(dashboard):** 9router-parity **Routing Strategy** settings card on Settings → Routing, plus a per-provider account-routing override on the provider detail page ([#6678](https://github.com/diegosouzapw/OmniRoute/pull/6678)) — surfaces the existing account round-robin / sticky-limit knobs and adds a new combo-level sticky round-robin (`comboStickyRoundRobinLimit`, resolved via `resolveComboStickyRoundRobinLimit()` — per-combo → global combo sticky → account sticky cascade) so combo targets can batch calls per target the same way account fallback already does. A new `providerStrategies` setting (Zod-validated map, `src/shared/validation/settingsSchemas.ts`) lets a specific provider override the global `fallbackStrategy`/`stickyRoundRobinLimit` without touching the account-wide default, wired into `getProviderCredentials()` (`src/sse/services/auth.ts`) ahead of the global fallback. Regression guard: `tests/unit/combo-rr-sticky-9router.test.ts`, `tests/unit/settings-ui-layout-static.test.ts`. (thanks @SeaXen)
|
||||
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
|
||||
@@ -202,7 +202,7 @@
|
||||
"src/app/(dashboard)/dashboard/endpoint/EndpointPageClient.tsx": 2612,
|
||||
"src/app/(dashboard)/dashboard/health/page.tsx": 1091,
|
||||
"src/app/(dashboard)/dashboard/playground/components/tabs/ApiTab.tsx": 847,
|
||||
"src/app/(dashboard)/dashboard/providers/[id]/ProviderDetailPageClient.tsx": 784,
|
||||
"src/app/(dashboard)/dashboard/providers/[id]/ProviderDetailPageClient.tsx": 786,
|
||||
"src/app/(dashboard)/dashboard/providers/[id]/components/ConnectionRow.tsx": 942,
|
||||
"src/app/(dashboard)/dashboard/providers/[id]/components/modals/AddApiKeyModal.tsx": 961,
|
||||
"src/app/(dashboard)/dashboard/providers/[id]/components/modals/EditConnectionModal.tsx": 1278,
|
||||
@@ -259,9 +259,10 @@
|
||||
"src/shared/services/cliRuntime.ts": 1110,
|
||||
"src/shared/validation/schemas.ts": 2523,
|
||||
"_rebaseline_2026_06_28_5275_correlation_id_extract": "Extraction of the safe CorrelationId subset of #5275 (hartmark) — request correlation id stored in call_logs (migration 109) and returned via the X-Correlation-Id response header, WITHOUT the combo/resilience or build/lazy-loading changes (those stay in #5275). Own growth: callLogs.ts 975->985 (correlation_id column on CallLogSummaryRow + read/map), usageHistory.ts 983->988 (correlationId metadata normalize), chat.ts 1575->1632 (withCorrelationId response wiring + combo-failure log carrying correlationId), chatHelpers.ts new 811 (withCorrelationId helper + reqId threading; was 791<cap pre-feature). Cohesive request/logging chokepoint wiring; structural shrink of chat.ts tracked in #3501.",
|
||||
"_rebaseline_2026_07_09_6678_routing_strategy_9router": "#6678 (SeaXen) — 9router-parity Routing Strategy settings card + per-provider/combo sticky-round-robin override. Own growth: ProviderDetailPageClient.tsx 784->786 (single ProviderAccountRoutingCard mount + import), auth.ts 2448->2458 (providerStrategies override resolution: fallbackStrategy/stickyRoundRobinLimit per-provider cascade in getProviderCredentials). Both additive, zero unrelated refactor; new UI/logic lives in new files (ProviderAccountRoutingCard.tsx, RoutingStrategyCard.tsx, rrState.ts::resolveComboStickyRoundRobinLimit). chat.ts value below reflects the current release tip (grown by other concurrent PRs, e.g. #6640), not this PR own change.",
|
||||
"src/sse/handlers/chat.ts": 1796,
|
||||
"src/sse/handlers/chatHelpers.ts": 876,
|
||||
"src/sse/services/auth.ts": 2448,
|
||||
"src/sse/services/auth.ts": 2458,
|
||||
"open-sse/executors/default.ts": 877,
|
||||
"open-sse/translator/request/openai-responses.ts": 902,
|
||||
"open-sse/executors/kiro.ts": 944,
|
||||
|
||||
@@ -112,6 +112,7 @@ import {
|
||||
recordStickyRoundRobinSuccess,
|
||||
getStickyWeightedExecutionKey,
|
||||
recordStickyWeightedSuccess,
|
||||
resolveComboStickyRoundRobinLimit,
|
||||
} from "./combo/rrState.ts";
|
||||
import {
|
||||
validateResponseQuality,
|
||||
@@ -898,10 +899,9 @@ export async function handleComboChat({
|
||||
let runtimeStickyTargets: ResolvedComboUnit[] = runtimeUnits;
|
||||
if (strategy === "round-robin") {
|
||||
const perComboStickyLimit = (config as Record<string, unknown>).stickyRoundRobinLimit;
|
||||
runtimeStickyLimit = clampStickyRoundRobinTargetLimit(
|
||||
perComboStickyLimit !== undefined && perComboStickyLimit !== null
|
||||
? perComboStickyLimit
|
||||
: (settings as Record<string, unknown> | null)?.stickyRoundRobinLimit
|
||||
runtimeStickyLimit = resolveComboStickyRoundRobinLimit(
|
||||
perComboStickyLimit,
|
||||
settings as Record<string, unknown> | null
|
||||
);
|
||||
const { startIndex, counter } = getStickyRoundRobinStartIndex(
|
||||
combo.name,
|
||||
@@ -2495,10 +2495,9 @@ async function handleRoundRobinCombo({
|
||||
// sticky batching for both account fallback and combo targets. Values <= 1 preserve
|
||||
// the historical one-request-per-target rotation.
|
||||
const perComboStickyLimit = (config as Record<string, unknown>).stickyRoundRobinLimit;
|
||||
const stickyLimit = clampStickyRoundRobinTargetLimit(
|
||||
perComboStickyLimit !== undefined && perComboStickyLimit !== null
|
||||
? perComboStickyLimit
|
||||
: (settings as Record<string, unknown> | null)?.stickyRoundRobinLimit
|
||||
const stickyLimit = resolveComboStickyRoundRobinLimit(
|
||||
perComboStickyLimit,
|
||||
settings as Record<string, unknown> | null
|
||||
);
|
||||
const stickyRoundRobinEnabled = stickyLimit > 1;
|
||||
// Exhaustion-aware sticky: if the currently sticky target is no longer
|
||||
|
||||
@@ -96,3 +96,22 @@ export function recordStickyWeightedSuccess(
|
||||
|
||||
weightedStickyTargets.set(comboName, { executionKey, successCount });
|
||||
}
|
||||
|
||||
/**
|
||||
* Sticky batch size for round-robin combo targets (9router parity).
|
||||
* Per-combo config → comboStickyRoundRobinLimit → stickyRoundRobinLimit.
|
||||
* Uses clampStickyRoundRobinTargetLimit defined above in this module.
|
||||
*/
|
||||
export function resolveComboStickyRoundRobinLimit(
|
||||
perComboLimit: unknown,
|
||||
settings: Record<string, unknown> | null | undefined
|
||||
): number {
|
||||
if (perComboLimit !== undefined && perComboLimit !== null) {
|
||||
return clampStickyRoundRobinTargetLimit(perComboLimit);
|
||||
}
|
||||
const comboSticky = settings?.comboStickyRoundRobinLimit;
|
||||
if (comboSticky !== undefined && comboSticky !== null) {
|
||||
return clampStickyRoundRobinTargetLimit(comboSticky);
|
||||
}
|
||||
return clampStickyRoundRobinTargetLimit(settings?.stickyRoundRobinLimit);
|
||||
}
|
||||
|
||||
@@ -45,6 +45,7 @@ import CustomModelsSection from "./components/CustomModelsSection";
|
||||
import ConnectionsListPanel from "./components/ConnectionsListPanel";
|
||||
import CoolingConnectionsPanel from "./components/CoolingConnectionsPanel";
|
||||
import ConnectionsHeaderToolbar from "./components/ConnectionsHeaderToolbar";
|
||||
import ProviderAccountRoutingCard from "../../settings/components/ProviderAccountRoutingCard";
|
||||
import ZedImportCard from "./components/ZedImportCard";
|
||||
import ProviderPageHeader from "./components/ProviderPageHeader";
|
||||
import CompatibleNodeCard from "./components/CompatibleNodeCard";
|
||||
@@ -488,6 +489,7 @@ export default function ProviderDetailPageClient() {
|
||||
)}
|
||||
{!isUpstreamProxyProvider && !isFreeNoAuth && (
|
||||
<Card>
|
||||
<ProviderAccountRoutingCard providerKey={providerId} connectionCount={connections.length} />
|
||||
<ConnectionsHeaderToolbar
|
||||
providerId={providerId}
|
||||
providerInfo={providerInfo}
|
||||
|
||||
@@ -0,0 +1,132 @@
|
||||
"use client";
|
||||
|
||||
import { useCallback, useEffect, useState } from "react";
|
||||
import { Input, Select } from "@/shared/components";
|
||||
import { useTranslations } from "next-intl";
|
||||
import { ACCOUNT_FALLBACK_STRATEGY_VALUES } from "@/shared/constants/routingStrategies";
|
||||
|
||||
type ProviderStrategyOverride = {
|
||||
fallbackStrategy?: string;
|
||||
stickyRoundRobinLimit?: number;
|
||||
};
|
||||
|
||||
type Props = {
|
||||
providerKey: string;
|
||||
connectionCount: number;
|
||||
};
|
||||
|
||||
const STRATEGY_OPTIONS = ACCOUNT_FALLBACK_STRATEGY_VALUES.filter((v) =>
|
||||
["fill-first", "round-robin", "priority", "p2c", "random", "least-used"].includes(v)
|
||||
);
|
||||
|
||||
function clampProviderStickyLimit(raw: string): number {
|
||||
const val = parseInt(raw, 10);
|
||||
return Math.min(10, Math.max(1, Number.isNaN(val) ? 3 : val));
|
||||
}
|
||||
|
||||
/** Loads/saves the per-provider account-routing override. Extracted out of the
|
||||
* component body to keep ProviderAccountRoutingCard's own render function small. */
|
||||
function useProviderAccountRoutingState(providerKey: string) {
|
||||
const [strategy, setStrategy] = useState<string>("");
|
||||
const [stickyLimit, setStickyLimit] = useState("3");
|
||||
const [busy, setBusy] = useState(false);
|
||||
|
||||
const load = useCallback(async () => {
|
||||
const res = await fetch("/api/settings", { cache: "no-store" });
|
||||
if (!res.ok) return;
|
||||
const data = await res.json();
|
||||
const override = ((data?.providerStrategies || {}) as Record<string, ProviderStrategyOverride>)[
|
||||
providerKey
|
||||
];
|
||||
setStrategy(override?.fallbackStrategy || "");
|
||||
setStickyLimit(
|
||||
override?.stickyRoundRobinLimit != null ? String(override.stickyRoundRobinLimit) : ""
|
||||
);
|
||||
}, [providerKey]);
|
||||
|
||||
useEffect(() => {
|
||||
load().catch(console.error);
|
||||
}, [load]);
|
||||
|
||||
const save = useCallback(
|
||||
async (nextStrategy: string, nextSticky: string) => {
|
||||
setBusy(true);
|
||||
try {
|
||||
const res = await fetch("/api/settings", { cache: "no-store" });
|
||||
if (!res.ok) throw new Error("Failed to fetch current settings");
|
||||
const data = await res.json();
|
||||
const current = (data?.providerStrategies || {}) as Record<
|
||||
string,
|
||||
ProviderStrategyOverride
|
||||
>;
|
||||
const override: ProviderStrategyOverride = {};
|
||||
if (nextStrategy) override.fallbackStrategy = nextStrategy;
|
||||
if (nextStrategy === "round-robin" && nextSticky !== "") {
|
||||
override.stickyRoundRobinLimit = clampProviderStickyLimit(nextSticky);
|
||||
}
|
||||
const updated = { ...current };
|
||||
if (Object.keys(override).length === 0) delete updated[providerKey];
|
||||
else updated[providerKey] = override;
|
||||
const patchRes = await fetch("/api/settings", {
|
||||
method: "PATCH",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ providerStrategies: updated }),
|
||||
});
|
||||
if (!patchRes.ok) throw new Error("Failed to save provider routing settings");
|
||||
} catch (e) {
|
||||
console.error(e);
|
||||
} finally {
|
||||
setBusy(false);
|
||||
}
|
||||
},
|
||||
[providerKey]
|
||||
);
|
||||
|
||||
return { strategy, setStrategy, stickyLimit, setStickyLimit, busy, save };
|
||||
}
|
||||
|
||||
export default function ProviderAccountRoutingCard({ providerKey, connectionCount }: Props) {
|
||||
const t = useTranslations("settings");
|
||||
const { strategy, setStrategy, stickyLimit, setStickyLimit, busy, save } =
|
||||
useProviderAccountRoutingState(providerKey);
|
||||
|
||||
if (connectionCount < 2) return null;
|
||||
|
||||
return (
|
||||
<div className="mb-4 rounded-lg border border-border/60 bg-black/[0.02] dark:bg-white/[0.02] p-3">
|
||||
<p className="text-sm font-medium">{t("providerAccountRoutingTitle")}</p>
|
||||
<p className="text-xs text-text-muted mb-3">{t("providerAccountRoutingDesc")}</p>
|
||||
<div className="grid grid-cols-1 sm:grid-cols-2 gap-3">
|
||||
<Select
|
||||
label={t("providerRoutingStrategy")}
|
||||
disabled={busy}
|
||||
value={strategy}
|
||||
onChange={(e) => {
|
||||
const v = e.target.value;
|
||||
setStrategy(v);
|
||||
save(v, stickyLimit).catch(console.error);
|
||||
}}
|
||||
>
|
||||
<option value="">{t("providerRoutingInheritGlobal")}</option>
|
||||
{STRATEGY_OPTIONS.map((opt) => (
|
||||
<option key={opt} value={opt}>
|
||||
{opt}
|
||||
</option>
|
||||
))}
|
||||
</Select>
|
||||
{strategy === "round-robin" && (
|
||||
<Input
|
||||
label={t("stickyLimit")}
|
||||
type="number"
|
||||
min={1}
|
||||
max={10}
|
||||
disabled={busy}
|
||||
value={stickyLimit || "3"}
|
||||
onChange={(e) => setStickyLimit(e.target.value)}
|
||||
onBlur={() => save(strategy, stickyLimit).catch(console.error)}
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,282 @@
|
||||
"use client";
|
||||
|
||||
import { useCallback, useEffect, useState } from "react";
|
||||
import { Card, Input, Toggle } from "@/shared/components";
|
||||
import { useTranslations } from "next-intl";
|
||||
|
||||
type RoutingSettings = {
|
||||
fallbackStrategy?: string;
|
||||
stickyRoundRobinLimit?: number | string;
|
||||
comboStrategy?: string;
|
||||
comboStickyRoundRobinLimit?: number | string;
|
||||
};
|
||||
|
||||
type Translate = ReturnType<typeof useTranslations>;
|
||||
|
||||
async function patchSettings(body: Record<string, unknown>) {
|
||||
const res = await fetch("/api/settings", {
|
||||
method: "PATCH",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
if (!res.ok) throw new Error("settings patch failed");
|
||||
return res.json();
|
||||
}
|
||||
|
||||
async function patchComboDefaultsStrategy(strategy: string) {
|
||||
const res = await fetch("/api/settings/combo-defaults", {
|
||||
method: "PATCH",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ comboDefaults: { strategy } }),
|
||||
});
|
||||
if (!res.ok) throw new Error("combo-defaults patch failed");
|
||||
}
|
||||
|
||||
function numericLimit(value: number | string | undefined, fallback: number): number {
|
||||
const n = typeof value === "number" ? value : parseInt(String(value), 10);
|
||||
return Number.isFinite(n) ? n : fallback;
|
||||
}
|
||||
|
||||
/** Loads/persists the routing-strategy settings and tracks loading/busy state.
|
||||
* Extracted out of the component body to keep RoutingStrategyCard's own render
|
||||
* function under the complexity/size gate. */
|
||||
function useRoutingStrategySettings() {
|
||||
const [settings, setSettings] = useState<RoutingSettings>({
|
||||
fallbackStrategy: "fill-first",
|
||||
stickyRoundRobinLimit: 3,
|
||||
comboStrategy: "fallback",
|
||||
comboStickyRoundRobinLimit: 1,
|
||||
});
|
||||
const [loading, setLoading] = useState(true);
|
||||
const [busy, setBusy] = useState(false);
|
||||
|
||||
const load = useCallback(async () => {
|
||||
setLoading(true);
|
||||
try {
|
||||
const res = await fetch("/api/settings", { cache: "no-store" });
|
||||
if (!res.ok) return;
|
||||
const data = await res.json();
|
||||
setSettings({
|
||||
fallbackStrategy: data?.fallbackStrategy || "fill-first",
|
||||
stickyRoundRobinLimit: data?.stickyRoundRobinLimit ?? 3,
|
||||
comboStrategy: data?.comboStrategy || "fallback",
|
||||
comboStickyRoundRobinLimit: data?.comboStickyRoundRobinLimit ?? 1,
|
||||
});
|
||||
} finally {
|
||||
setLoading(false);
|
||||
}
|
||||
}, []);
|
||||
|
||||
useEffect(() => {
|
||||
load().catch(console.error);
|
||||
}, [load]);
|
||||
|
||||
const run = useCallback(async (fn: () => Promise<void>) => {
|
||||
setBusy(true);
|
||||
try {
|
||||
await fn();
|
||||
} catch (e) {
|
||||
console.error(e);
|
||||
} finally {
|
||||
setBusy(false);
|
||||
}
|
||||
}, []);
|
||||
|
||||
return { settings, setSettings, loading, busy, run };
|
||||
}
|
||||
|
||||
type SectionProps = {
|
||||
t: Translate;
|
||||
busy: boolean;
|
||||
settings: RoutingSettings;
|
||||
setSettings: React.Dispatch<React.SetStateAction<RoutingSettings>>;
|
||||
run: (fn: () => Promise<void>) => Promise<void>;
|
||||
};
|
||||
|
||||
function AccountRoundRobinSection({ t, busy, settings, setSettings, run }: SectionProps) {
|
||||
const accountRoundRobin = settings.fallbackStrategy === "round-robin";
|
||||
return (
|
||||
<>
|
||||
<div className="flex items-start sm:items-center justify-between gap-4">
|
||||
<div className="flex-1 min-w-0">
|
||||
<p className="font-medium text-sm sm:text-base">{t("accountRoundRobin")}</p>
|
||||
<p className="text-xs sm:text-sm text-text-muted">{t("accountRoundRobinDesc")}</p>
|
||||
</div>
|
||||
<Toggle
|
||||
checked={accountRoundRobin}
|
||||
disabled={busy}
|
||||
onChange={() =>
|
||||
run(async () => {
|
||||
const next = accountRoundRobin ? "fill-first" : "round-robin";
|
||||
const updated = await patchSettings({ fallbackStrategy: next });
|
||||
setSettings((prev) => ({ ...prev, fallbackStrategy: updated.fallbackStrategy || next }));
|
||||
})
|
||||
}
|
||||
/>
|
||||
</div>
|
||||
|
||||
{accountRoundRobin && (
|
||||
<div className="flex items-start sm:items-center justify-between gap-4 pt-2 border-t border-border/50">
|
||||
<div className="flex-1 min-w-0">
|
||||
<p className="font-medium text-sm sm:text-base">{t("stickyLimit")}</p>
|
||||
<p className="text-xs sm:text-sm text-text-muted">{t("stickyLimitDesc")}</p>
|
||||
</div>
|
||||
<Input
|
||||
type="number"
|
||||
min={1}
|
||||
max={10}
|
||||
disabled={busy}
|
||||
className="w-16 sm:w-20 text-center shrink-0"
|
||||
value={settings.stickyRoundRobinLimit ?? 3}
|
||||
onChange={(e) => setSettings((prev) => ({ ...prev, stickyRoundRobinLimit: e.target.value }))}
|
||||
onBlur={() =>
|
||||
run(async () => {
|
||||
const limit = Math.min(
|
||||
10,
|
||||
Math.max(1, parseInt(String(settings.stickyRoundRobinLimit), 10) || 3)
|
||||
);
|
||||
const updated = await patchSettings({ stickyRoundRobinLimit: limit });
|
||||
setSettings((prev) => ({
|
||||
...prev,
|
||||
stickyRoundRobinLimit: updated.stickyRoundRobinLimit ?? limit,
|
||||
}));
|
||||
})
|
||||
}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
function ComboRoundRobinSection({ t, busy, settings, setSettings, run }: SectionProps) {
|
||||
const comboRoundRobin = settings.comboStrategy === "round-robin";
|
||||
return (
|
||||
<>
|
||||
<div className="flex items-start sm:items-center justify-between gap-4 pt-4 border-t border-border/50">
|
||||
<div className="flex-1 min-w-0">
|
||||
<p className="font-medium text-sm sm:text-base">{t("comboRoundRobin")}</p>
|
||||
<p className="text-xs sm:text-sm text-text-muted">{t("comboRoundRobinDesc")}</p>
|
||||
</div>
|
||||
<Toggle
|
||||
checked={comboRoundRobin}
|
||||
disabled={busy}
|
||||
onChange={() =>
|
||||
run(async () => {
|
||||
const next = comboRoundRobin ? "fallback" : "round-robin";
|
||||
const defaultComboStrategy = next === "round-robin" ? "round-robin" : "priority";
|
||||
await patchComboDefaultsStrategy(defaultComboStrategy);
|
||||
const updated = await patchSettings({ comboStrategy: next });
|
||||
setSettings((prev) => ({ ...prev, comboStrategy: updated.comboStrategy || next }));
|
||||
})
|
||||
}
|
||||
/>
|
||||
</div>
|
||||
|
||||
{comboRoundRobin && (
|
||||
<div className="flex items-center justify-between pt-2 border-t border-border/50">
|
||||
<div>
|
||||
<p className="font-medium">{t("comboStickyLimit")}</p>
|
||||
<p className="text-sm text-text-muted">{t("comboStickyLimitDesc")}</p>
|
||||
</div>
|
||||
<Input
|
||||
type="number"
|
||||
min={1}
|
||||
max={100}
|
||||
disabled={busy}
|
||||
className="w-20 text-center"
|
||||
value={settings.comboStickyRoundRobinLimit ?? 1}
|
||||
onChange={(e) =>
|
||||
setSettings((prev) => ({ ...prev, comboStickyRoundRobinLimit: e.target.value }))
|
||||
}
|
||||
onBlur={() =>
|
||||
run(async () => {
|
||||
const limit = Math.min(
|
||||
100,
|
||||
Math.max(1, parseInt(String(settings.comboStickyRoundRobinLimit), 10) || 1)
|
||||
);
|
||||
const updated = await patchSettings({ comboStickyRoundRobinLimit: limit });
|
||||
setSettings((prev) => ({
|
||||
...prev,
|
||||
comboStickyRoundRobinLimit: updated.comboStickyRoundRobinLimit ?? limit,
|
||||
}));
|
||||
})
|
||||
}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
function RoutingSummaryFooter({
|
||||
t,
|
||||
accountRoundRobin,
|
||||
comboRoundRobin,
|
||||
accountStickyDisplay,
|
||||
comboStickyDisplay,
|
||||
}: {
|
||||
t: Translate;
|
||||
accountRoundRobin: boolean;
|
||||
comboRoundRobin: boolean;
|
||||
accountStickyDisplay: number;
|
||||
comboStickyDisplay: number;
|
||||
}) {
|
||||
return (
|
||||
<p className="text-xs text-text-muted italic pt-2 border-t border-border/50">
|
||||
{accountRoundRobin
|
||||
? t("routingStrategyAccountSummary", { limit: accountStickyDisplay })
|
||||
: t("routingStrategyFillFirstSummary")}
|
||||
{comboRoundRobin
|
||||
? t("routingStrategyComboSummary", { limit: comboStickyDisplay })
|
||||
: t("routingStrategyComboFallbackSummary")}
|
||||
</p>
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* 9router-style Routing Strategy card (Profile → Routing Strategy parity).
|
||||
* Settings → Routing, first card on the page.
|
||||
*/
|
||||
export default function RoutingStrategyCard() {
|
||||
const t = useTranslations("settings");
|
||||
const tc = useTranslations("common");
|
||||
const { settings, setSettings, loading, busy, run } = useRoutingStrategySettings();
|
||||
|
||||
const accountRoundRobin = settings.fallbackStrategy === "round-robin";
|
||||
const comboRoundRobin = settings.comboStrategy === "round-robin";
|
||||
const accountStickyDisplay = numericLimit(settings.stickyRoundRobinLimit, 3);
|
||||
const comboStickyDisplay = numericLimit(settings.comboStickyRoundRobinLimit, 1);
|
||||
|
||||
return (
|
||||
<Card>
|
||||
<div className="flex items-center gap-3 mb-4">
|
||||
<div className="p-2 rounded-lg bg-blue-500/10 text-blue-500 shrink-0">
|
||||
<span className="material-symbols-outlined text-[20px]" aria-hidden="true">
|
||||
route
|
||||
</span>
|
||||
</div>
|
||||
<div>
|
||||
<h3 className="text-base sm:text-lg font-semibold">{t("routingStrategyTitle")}</h3>
|
||||
<p className="text-xs text-text-muted">{t("routingStrategySubtitle")}</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{loading ? (
|
||||
<p className="text-sm text-text-muted">{tc("loading")}</p>
|
||||
) : (
|
||||
<div className="flex flex-col gap-4">
|
||||
<AccountRoundRobinSection t={t} busy={busy} settings={settings} setSettings={setSettings} run={run} />
|
||||
<ComboRoundRobinSection t={t} busy={busy} settings={settings} setSettings={setSettings} run={run} />
|
||||
<RoutingSummaryFooter
|
||||
t={t}
|
||||
accountRoundRobin={accountRoundRobin}
|
||||
comboRoundRobin={comboRoundRobin}
|
||||
accountStickyDisplay={accountStickyDisplay}
|
||||
comboStickyDisplay={comboStickyDisplay}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
</Card>
|
||||
);
|
||||
}
|
||||
@@ -1,6 +1,7 @@
|
||||
"use client";
|
||||
|
||||
import { useTranslations } from "next-intl";
|
||||
import RoutingStrategyCard from "../components/RoutingStrategyCard";
|
||||
import RoutingTab from "../components/RoutingTab";
|
||||
import ModelRoutingSection from "@/shared/components/ModelRoutingSection";
|
||||
import ComboDefaultsTab from "../components/ComboDefaultsTab";
|
||||
@@ -13,6 +14,7 @@ export default function SettingsRoutingPage() {
|
||||
return (
|
||||
<div className="space-y-6">
|
||||
<p className="text-sm text-text-muted">{t("routingSettingsIntro")}</p>
|
||||
<RoutingStrategyCard />
|
||||
<ComboDefaultsTab />
|
||||
<ModelAliasesUnified />
|
||||
<FallbackChainsEditor />
|
||||
|
||||
@@ -5326,6 +5326,22 @@
|
||||
"strictRandomDesc": "Shuffle deck — uses each account once before reshuffling",
|
||||
"stickyLimit": "Sticky Limit",
|
||||
"stickyLimitDesc": "Calls per account before switching when global account fallback uses Round Robin",
|
||||
"routingStrategyTitle": "Routing Strategy",
|
||||
"routingStrategySubtitle": "Match 9router: account round-robin, sticky limits, and combo rotation",
|
||||
"accountRoundRobin": "Round Robin",
|
||||
"accountRoundRobinDesc": "Cycle through accounts to distribute load",
|
||||
"comboRoundRobin": "Combo Round Robin",
|
||||
"comboRoundRobinDesc": "Cycle through combo targets instead of always starting with the first",
|
||||
"comboStickyLimit": "Combo Sticky Limit",
|
||||
"comboStickyLimitDesc": "Calls per combo target before switching",
|
||||
"routingStrategyAccountSummary": "Distributing requests across accounts with {limit} calls per account.",
|
||||
"routingStrategyFillFirstSummary": "Using accounts in priority order (Fill First).",
|
||||
"routingStrategyComboSummary": " Combos rotate after {limit} call(s) per target.",
|
||||
"routingStrategyComboFallbackSummary": " Combos use each combo's configured strategy (default priority/fallback).",
|
||||
"providerAccountRoutingTitle": "Multi-account routing",
|
||||
"providerAccountRoutingDesc": "Override global account strategy for this provider (9router parity).",
|
||||
"providerRoutingStrategy": "Account strategy",
|
||||
"providerRoutingInheritGlobal": "Inherit global default",
|
||||
"modelAliases": "Model Aliases",
|
||||
"modelAliasesTitle": "Model Aliases",
|
||||
"modelAliasesDesc": "Remap model names using exact matches or wildcard patterns.",
|
||||
|
||||
@@ -97,6 +97,9 @@ export async function getSettings() {
|
||||
tailscaleUrl: "",
|
||||
stickyRoundRobinLimit: 3,
|
||||
disableSessionStickiness: false,
|
||||
comboStrategy: "fallback",
|
||||
comboStickyRoundRobinLimit: 1,
|
||||
providerStrategies: {},
|
||||
requestRetry: 3,
|
||||
maxRetryIntervalSec: 30,
|
||||
antigravitySignatureCacheMode: "enabled",
|
||||
|
||||
@@ -183,6 +183,18 @@ export const updateSettingsSchema = z.object({
|
||||
fallbackStrategy: z.enum(ACCOUNT_FALLBACK_STRATEGY_VALUES).optional(),
|
||||
wildcardAliases: z.array(z.object({ pattern: z.string(), target: z.string() })).optional(),
|
||||
stickyRoundRobinLimit: z.number().int().min(0).max(1000).optional(),
|
||||
/** 9router parity: global combo expansion strategy (fallback vs round-robin). */
|
||||
comboStrategy: z.enum(["fallback", "round-robin"]).optional(),
|
||||
comboStickyRoundRobinLimit: z.number().int().min(1).max(100).optional(),
|
||||
providerStrategies: z
|
||||
.record(
|
||||
z.string().trim().min(1),
|
||||
z.object({
|
||||
fallbackStrategy: z.enum(ACCOUNT_FALLBACK_STRATEGY_VALUES).optional(),
|
||||
stickyRoundRobinLimit: z.number().int().min(1).max(1000).optional(),
|
||||
})
|
||||
)
|
||||
.optional(),
|
||||
// #6168: global session-stickiness opt-out (per-combo config overrides this).
|
||||
disableSessionStickiness: z.boolean().optional(),
|
||||
requestRetry: z.number().int().min(0).max(10).optional(),
|
||||
|
||||
@@ -1477,7 +1477,13 @@ export async function getProviderCredentials(
|
||||
|
||||
const orderedConnections = withQuota;
|
||||
|
||||
const strategy = settings.fallbackStrategy || "fill-first";
|
||||
const providerStrategyOverrides = (settings.providerStrategies || {}) as Record<
|
||||
string,
|
||||
{ fallbackStrategy?: string; stickyRoundRobinLimit?: number }
|
||||
>;
|
||||
const providerOverride = providerStrategyOverrides[resolvedId] || {};
|
||||
const strategy =
|
||||
providerOverride.fallbackStrategy || settings.fallbackStrategy || "fill-first";
|
||||
|
||||
let connection;
|
||||
const affinityConnection = await selectSessionAffinityConnection(
|
||||
@@ -1498,7 +1504,11 @@ export async function getProviderCredentials(
|
||||
if (connection) {
|
||||
// Session affinity selected a connection before global sticky routing.
|
||||
} else if (strategy === "round-robin") {
|
||||
const stickyLimit = toNumber((settings as Record<string, unknown>).stickyRoundRobinLimit, 3);
|
||||
const stickyLimit = toNumber(
|
||||
providerOverride.stickyRoundRobinLimit ??
|
||||
(settings as Record<string, unknown>).stickyRoundRobinLimit,
|
||||
3
|
||||
);
|
||||
|
||||
// If excluding account(s) (fallback scenario), skip sticky logic and go straight to LRU.
|
||||
// This prevents same-model retries from getting stuck on a failed account.
|
||||
|
||||
@@ -139,6 +139,7 @@
|
||||
"tests/unit/combo-roundrobin-compat-fallback-6238.test.ts",
|
||||
"tests/unit/combo-routing-engine.test.ts",
|
||||
"tests/unit/combo-rr-fallback-advance-948.test.ts",
|
||||
"tests/unit/combo-rr-sticky-9router.test.ts",
|
||||
"tests/unit/combo-scoring-inspector.test.ts",
|
||||
"tests/unit/combo-selected-connection-success.test.ts",
|
||||
"tests/unit/combo-sessionless-pin-3825.test.ts",
|
||||
|
||||
13
tests/unit/combo-rr-sticky-9router.test.ts
Normal file
13
tests/unit/combo-rr-sticky-9router.test.ts
Normal file
@@ -0,0 +1,13 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { resolveComboStickyRoundRobinLimit } from "../../open-sse/services/combo/rrState.ts";
|
||||
|
||||
test("resolveComboStickyRoundRobinLimit prefers per-combo, then comboSticky, then account sticky", () => {
|
||||
const settings = { stickyRoundRobinLimit: 3, comboStickyRoundRobinLimit: 2 };
|
||||
assert.equal(resolveComboStickyRoundRobinLimit(5, settings), 5);
|
||||
assert.equal(resolveComboStickyRoundRobinLimit(undefined, settings), 2);
|
||||
assert.equal(
|
||||
resolveComboStickyRoundRobinLimit(undefined, { stickyRoundRobinLimit: 7 }),
|
||||
7
|
||||
);
|
||||
});
|
||||
@@ -99,6 +99,7 @@ test("Global Routing page renders top-level modules in the requested order", ()
|
||||
const routingTab = readSrc("src/app/(dashboard)/dashboard/settings/components/RoutingTab.tsx");
|
||||
|
||||
assertInOrder(page, [
|
||||
"<RoutingStrategyCard",
|
||||
"<ComboDefaultsTab",
|
||||
"<ModelAliasesUnified",
|
||||
"<FallbackChainsEditor",
|
||||
|
||||
Reference in New Issue
Block a user