feat(quota): serializa concorrência por conexão no caminho quota-share (FASE 2.1) (#4970)

O gating de quota-share em selectQuotaShareTarget é fail-open: uma conexão
at-cap só é despriorizada, nunca bloqueada. Com 1 conexão por conta de
assinatura (caso comum), chamadas concorrentes ainda floodam a conta (→ 429 +
cooldown) — provado live na .15: 3 chamadas concorrentes com max_concurrent=1
despacharam todas em 94ms.

Adiciona um semáforo POR CONEXÃO em torno do dispatch quota-share: chamadas
excedentes esperam na fila em vez de floodar (key qsconn:<connectionId>, cap =
max_concurrent da conexão). Fail-open em fila saturada/timeout para nunca
piorar disponibilidade. Gated por strategy===quota-share + kill-switch
resilienceSettings.quotaShareConcurrencyLimit (default on; UI no ResilienceTab).

Lógica extraível isolada no leaf puro combo/quotaShareConcurrency.ts
(unit-testado: estabilidade da key, no-op sem cap, serialização real,
fail-open). Settings + schema + UI espelham comboCooldownWait.

Co-authored-by: Diego Rodrigues de Sa e Souza <souzamiriamrodrigues790@gmail.com>
This commit is contained in:
Diego Rodrigues de Sa e Souza
2026-06-24 23:54:43 -03:00
committed by GitHub
parent 028ed4ea7b
commit 1f685ebcd7
10 changed files with 458 additions and 6 deletions

View File

@@ -3,6 +3,7 @@
"_rebaseline_2026_06_24_4945_task_aware_router_precedence": "Base-red fix: the #4945 task-aware reordering (reorderByTaskWeight) runs for strategy \"auto\" and was applied AFTER the auto router (selectWithStrategy: lkgp/cost/etc) had already pinned orderedTargets[0], silently defeating the operator's explicit LKGP/cost choice (proven: tests/unit/combo-routing-engine.test.ts 'honors LKGP after filtering to tool-capable models' + 'falls back to the full pool when tool filtering empties candidates' selected gpt-oss-120b — the 1st target — instead of the LKGP/cost winner; instrumentation showed orderedTargets[0]=claude post-filter then gpt-oss post-task-reorder). open-sse/services/combo.ts 3306->3321 (+15 = a one-line `autoUsedExplicitRouter` flag set when the explicit router succeeds, and at the task-aware step: when that flag is set, pin orderedTargets[0] and let task-aware refine only the fallback tail — reorderByTaskWeight returns the same objects so identity filtering is safe). gpt-oss-120b correctly REMAINS tool-capable (model-capabilities-registry.test.ts asserts true), so this is NOT a tool-capability change. Irreducible chokepoint wiring at the existing auto-route boundary; not extractable. Covered by the two combo-routing-engine.test.ts cases above + combo-task-aware.test.ts (task-aware still active for non-router strategies).",
"_rebaseline_2026_06_24_per_connection_max_concurrent": "Feature (per-connection max_concurrent enforcement, issue #9 follow-up) own growth: open-sse/services/combo.ts 3225->3238 (+13 net). Provider connections already carry a maxConcurrent ceiling (provider_connections.max_concurrent, migration 029, set in the UI/API/DB) but routing ignored it, so low-concurrency subscription accounts (GLM/MiniMax ~1) got flooded -> 429 + cooldown, worse under quota-share where keys share one account. The new wiring lives at the two combo dispatch chokepoints: (1) the strategy===\"quota-share\" branch resolves each target connection's cap and passes it into selectQuotaShareTarget so an at-cap connection is deprioritized (fail-open, never hard-blocked); (2) handleRoundRobinCombo resolves the target connection's cap and uses it as the semaphore maxConcurrency (fallback = combo-level concurrency). ALL extractable logic was moved into the new pure leaf open-sse/services/combo/concurrencyCaps.ts (resolveMaxConcurrentByConnection + makeConnectionConcurrencyResolver, DB reads deduped per connectionId, fail-open) and the pure effectiveMaxConcurrency resolver into combo/comboPredicates.ts; combo.ts keeps only the irreducible chokepoint wiring (a 5-line import + the two call-site bindings). Cohesive at the existing dispatch boundaries, not further extractable. Structural shrink of combo.ts tracked in #3501. Covered by tests/unit/quota-share-strategy.test.ts (per-connection maxConcurrent gating cases) + tests/unit/combo/effective-max-concurrency.test.ts.",
"_rebaseline_2026_06_24_combo_cooldown_wait_quota_share": "Feature quota-share combo cooldown-aware retry (Variante A) own growth: open-sse/services/combo.ts 3225->3293 (+68 = the cooldown-wait wrap inside handleComboChat's quota-share path. The existing setTry loop body is LEFT at its original indentation: instead of an outer `while (true)` (which would re-indent ~1600 lines and bloat the review), the setTry loop is hoisted into a small recursive closure `dispatchWithCooldownRetry`, and a wait+redispatch is a tail `return dispatchWithCooldownRetry()` — re-running ONLY the set loop (exactly the prior continue-to-top-of-set-loop semantics) while selection/shadow-routing/setup above stay untouched. At the 429 crystallization point the lock reason is resolved via getModelLockoutInfo, the decision via the new pure resolveComboCooldownWaitDecision, then await waitForCooldownAwareRetry (499 on abort), decrement the budget, recurse. Gated to strategy==='quota-share' && comboCooldownWait.enabled. `git diff` == `git diff -w` for combo.ts (zero re-indentation noise). The gating policy + reason resolution are extracted to the new pure leaf open-sse/services/combo/comboCooldownRetry.ts (<cap, unit-tested) to keep combo.ts thin; only the wait orchestration (one getModelLockoutInfo lookup + one await) stays at the chokepoint, not extractable. The quota_exhausted/auth/not-found exclusions defend against isRetryableModelLockoutReason (auth.ts:533) treating quota_exhausted as retryable (locked-until-midnight). Covered by tests/unit/combo-cooldown-retry.test.ts (pure helper, all branches) + tests/unit/combo-quota-share-cooldown-wait.test.ts (integration: rate_limit waits+recovers, quota_exhausted no-wait, abort=499, non-quota-share unchanged) + tests/unit/resilience-settings-combo-cooldown-wait.test.ts (settings round-trip/clamp). Also src/app/(dashboard)/dashboard/settings/components/ResilienceTab.tsx 983->1098 (+115 = the ComboCooldownWaitCard exposing enabled/maxWaitMs/maxAttempts/budgetMs in Settings > Resilience, mirroring WaitForCooldownCard; wired through GET/PATCH in src/app/api/resilience/route.ts + comboCooldownWaitSettingsSchema in src/shared/validation/schemas/settings.ts; new UI labels use the t(key)||English-fallback pattern, en-only). Structural shrink of combo.ts + ResilienceTab tracked in #3501.",
"_rebaseline_2026_06_24_quota_share_concurrency_limit": "Feature FASE 2.1 (per-connection concurrency limit for quota-share combos) own growth. The quota-share gating in selectQuotaShareTarget is FAIL-OPEN (an at-cap connection is only deprioritized, never hard-blocked), so with a single-connection subscription-account pool concurrent requests still flood the account — empirically proven on the .15 deploy: 3 concurrent share-key calls to one minimax connection with max_concurrent=1 were all dispatched within 94ms. This adds a per-CONNECTION semaphore around the quota-share dispatch so excess concurrent requests WAIT in the queue instead of flooding (key = qsconn:<connectionId>, cap = the connection's max_concurrent; fail-open on a saturated queue/timeout to never worsen availability). open-sse/services/combo.ts 3306->3340 (+34 = the irreducible chokepoint wiring: the quotaShareConcurrencyEnabled gate, the acquire (lookupPositiveCap + acquireQuotaShareConcurrencySlot) around dispatchWithCooldownRetry, the release in the outer finally, and the updated cooldown-wait comment). ALL extractable logic lives in the new pure leaf open-sse/services/combo/quotaShareConcurrency.ts (<cap, unit-tested: key stability, no-cap/empty no-op, genuine serialization, fail-open). src/lib/resilience/settings.ts ->841 (+41 = QuotaShareConcurrencyLimitSettings interface + default {enabled:true} + normalizeQuotaShareConcurrencyLimitSettings + resolve/merge/legacy-fallback wiring; crossed the 800 new-file cap, mirrors the existing comboCooldownWait block). src/app/(dashboard)/dashboard/settings/components/ResilienceTab.tsx 1098->1183 (+85 = QuotaShareConcurrencyLimitCard, a kill-switch toggle mirroring ComboCooldownWaitCard; wired through GET/PATCH in src/app/api/resilience/route.ts + quotaShareConcurrencyLimitSettingsSchema in src/shared/validation/schemas/settings.ts; t(key)||English-fallback, en-only). Covered by tests/unit/combo/quota-share-concurrency.test.ts + tests/unit/resilience-settings-quota-share-concurrency.test.ts. Structural shrink of combo.ts + ResilienceTab tracked in #3501.",
"_rebaseline_2026_06_23_4774_combo_legacy_strip": "PR #4774 (KooshaPari, #4382 round-trip) own growth: src/app/(dashboard)/dashboard/combos/page.tsx 4434->4456 (+22 = the client-side LEGACY_COMBO_RESILIENCE_KEYS Set gains queueTimeoutMs + the 12 v3.8.31-era removed keys (queueDepth/fallbackDelayMs/handoffProviders/maxComboDepth/manifestRouting/complexityAwareRouting/pipeline_enabled/pipelineConcurrency/shadowRouting/evalRouting/resetAwareEnabled/resetAwareWindow) with explanatory comments, mirroring the server-side strip list in src/app/api/combos/[id]/route.ts so the modal never re-introduces removed keys on Save). Functional strip list, not a movable block; combos/page.tsx structural shrink tracked in #3501. Covered by tests/unit/combo-config.test.ts (auto-promote + passthrough + legacy-key round-trip).",
"_rebaseline_2026_06_21_4421_node_lookup": "Issue #4421 own growth: src/lib/db/providers.ts 1050->1063 (+13 = resolveProviderNodeForConnection at the existing provider-node lookup — resolves a connection node by exact id OR the bare derived type when unambiguous, + import). Pure selection logic in the new src/lib/db/providerNodeSelect.ts (<cap, unit-tested); cohesive DB wiring next to getProviderNodeById, not extractable. Covered by tests/unit/provider-node-select-4421.test.ts.",
"_rebaseline_2026_06_21_v3833_release_basered_fixes": "Release v3.8.33: 5 deterministic base-reds inherited from parallel-session merges (fast-gates PR→release não rodam test:unit completo) consertados no PR de release. src/sse/services/auth.ts 2279->2289 (+10): #4530 só fiou maxCooldownMs nos 3 sites de combo.ts; os 4 sites de markAccountUnavailable (per-model quota, grok-web 403, per-model 403, local 404) nunca passavam o cap → resolvo mlSettings uma vez e passo maxCooldownMs em todos. tests/unit/db-core-init.test.ts 864->867 (+3): comentário explicando o cap intencional busy_timeout 5s->2s do v3.8.32. Wiring necessário ao chokepoint de lockout; não extraível. Coberto por model-lockout-max-cooldown.test.ts + db-core-init.test.ts. (Demais base-reds — áudio mp3 #912/#913 dedup de handler em geminiHelper.ts, e closure #3578 src/models/ no package.json files — não cresceram arquivo congelado.)",
@@ -146,7 +147,7 @@
"_rebaseline_2026_06_24_headroom_strategy": "Headroom-aware connection selection (dario technique): combo.ts 3168->3180 (+12 = a new `else if (strategy === \"headroom\")` dispatch branch in handleComboChat that delegates to orderTargetsByHeadroom + its log line, plus the import). The actual logic lives OUT of the god-file: the pure ranker rankByHeadroom/computeHeadroom is the new leaf open-sse/services/combo/headroomRanking.ts (91 LOC, <cap) and the async orderer orderTargetsByHeadroom is appended to the existing open-sse/services/combo/quotaStrategies.ts (<cap) next to its sibling reset-aware/reset-window orderers (reuses their connection-expansion machinery). headroom = 1 - max(util_5h, util_7d) from getSaturation (src/lib/quota/saturationSignals.ts), prefers the connection with the most free capacity. Only the dispatch wiring is irreducible at the existing combo strategy chokepoint (mirrors the reset-aware/reset-window/context-optimized branches); not extractable without hiding the call site. fill-first stays default; all existing strategies untouched. Covered by tests/unit/combo-headroom-ranking.test.ts (pure helper) + tests/unit/combo-headroom-strategy.test.ts (orderer, saturation injected). Structural shrink of combo.ts tracked in #3501.",
"_rebaseline_2026_06_24_quota_share_strategy": "Dedicated quota-share strategy (Phase 3 #9): combo.ts 3180->3190 (+10 = one new `else if (strategy === \"quota-share\")` dispatch branch in handleComboChat that delegates 100% to selectQuotaShareTarget + its log line, plus the import). All the new logic lives OUT of the god-file in two new leaves under open-sse/services/combo/: quotaShareInflight.ts (in-flight counter with TTL/lease, ~150 LOC <cap) and quotaShareStrategy.ts (per-model bucket gating via isBucketSaturated + DRR proportional to weight + P2C over in-flight, ~240 LOC <cap). Only the dispatch wiring is irreducible at the existing combo strategy chokepoint (mirrors the headroom/reset-aware/reset-window/context-optimized branches); not extractable without hiding the call site. ZERO existing strategy cases were modified — only this branch was added, and the qtSd/ combos switched from fill-first to quota-share in src/lib/quota/quotaCombos.ts. Covered by tests/unit/quota-share-strategy.test.ts (gating, DRR fairness, P2C in-flight, fail-open, activation). Structural shrink of combo.ts tracked in #3501.",
"_rebaseline_2026_06_24_task_aware_routing": "Task-aware routing strategy (port PR #2045, OmniRoute #4945): combo.ts 3190->3225 (+35) = one new `else if (strategy === \"task-aware\")` dispatch branch delegating 100% to selectTaskAwareTarget + its imports/log lines. All scoring/classification logic lives OUT of the god-file in the new leaf open-sse/services/taskAwareRouting.ts (553 LOC <cap). Only the dispatch wiring is irreducible at the existing combo strategy chokepoint (mirrors quota-share/headroom/reset-aware branches). ZERO existing strategy cases modified. Covered by tests/unit/combo-task-aware.test.ts (35 tests). Structural shrink of combo.ts tracked in #3501.",
"open-sse/services/combo.ts": 3321,
"open-sse/services/combo.ts": 3355,
"open-sse/services/compression/strategySelector.ts": 848,
"open-sse/services/rateLimitManager.ts": 1035,
"open-sse/services/tokenRefresh.ts": 2070,
@@ -186,7 +187,7 @@
"src/app/(dashboard)/dashboard/settings/components/MemorySkillsTab.tsx": 898,
"src/app/(dashboard)/dashboard/settings/components/PricingTab.tsx": 1012,
"src/app/(dashboard)/dashboard/settings/components/ProxyRegistryManager.tsx": 1089,
"src/app/(dashboard)/dashboard/settings/components/ResilienceTab.tsx": 1098,
"src/app/(dashboard)/dashboard/settings/components/ResilienceTab.tsx": 1183,
"src/app/(dashboard)/dashboard/settings/components/RoutingTab.tsx": 1629,
"src/app/(dashboard)/dashboard/settings/components/SystemStorageTab.tsx": 1924,
"src/app/(dashboard)/dashboard/usage/components/BudgetTab.tsx": 1016,
@@ -210,6 +211,7 @@
"src/lib/memory/retrieval.ts": 1171,
"src/lib/modelsDevSync.ts": 934,
"src/lib/providers/validation.ts": 4523,
"src/lib/resilience/settings.ts": 841,
"src/lib/tailscaleTunnel.ts": 1202,
"src/lib/usage/callLogs.ts": 975,
"src/lib/usage/providerLimits.ts": 950,

View File

@@ -79,7 +79,9 @@ import { selectQuotaShareTarget } from "./combo/quotaShareStrategy.ts";
import {
resolveMaxConcurrentByConnection,
makeConnectionConcurrencyResolver,
lookupPositiveCap,
} from "./combo/concurrencyCaps.ts";
import { acquireQuotaShareConcurrencySlot } from "./combo/quotaShareConcurrency.ts";
import { orderTargetsByEvalScores } from "./evalRouting.ts";
import { generateRoutingHints } from "./manifestAdapter";
import type { RoutingHint } from "./manifestAdapter";
@@ -1667,9 +1669,11 @@ export async function handleComboChat({
// because the target hit a SHORT transient cooldown, we wait it out and
// re-run the whole set loop instead of propagating the 429. `globalAttempts`
// persists across these waits so MAX_GLOBAL_ATTEMPTS still bounds total work.
// The wait happens at the crystallization point, which holds no semaphore slot
// (the quota-share path never calls semaphore.acquire) and no explicit
// in-flight lease that must be released here.
// The wait happens at the crystallization point. The only semaphore slot the
// quota-share path may hold is the FASE 2.1 per-connection concurrency slot
// (acquired once around dispatchWithCooldownRetry below); it is intentionally
// kept across the wait so the account stays "busy", and is released by the
// outer finally — not here.
//
// The set loop is wrapped in a small recursive closure rather than an extra
// labelled `while (true)` so the loop body keeps its original indentation; a
@@ -1682,6 +1686,14 @@ export async function handleComboChat({
let comboCooldownAttempt = 0;
let comboCooldownBudgetLeftMs = resilienceSettings.comboCooldownWait.budgetMs;
// FASE 2.1: per-connection concurrency limit for quota-share. The gating in
// selectQuotaShareTarget is fail-open and cannot hard-limit a single-connection
// pool, so we serialize concurrent requests to the selected account through a
// per-connection semaphore. Enabled only for quota-share combos (the cap is the
// account's) and gated by the kill-switch; the slot wraps the whole dispatch.
const quotaShareConcurrencyEnabled =
strategy === "quota-share" && resilienceSettings.quotaShareConcurrencyLimit.enabled;
const dispatchWithCooldownRetry = async (): Promise<Response> => {
for (let setTry = 0; setTry <= maxSetRetries; setTry++) {
// #1731: Per-set-iteration set of providers whose quota is fully exhausted.
@@ -2658,9 +2670,31 @@ export async function handleComboChat({
return errorResponse(503, "Combo routing completed without an upstream response");
};
// FASE 2.1: acquire the per-connection concurrency slot for the selected
// quota-share target once, around the whole dispatch (including any
// cooldown-aware re-dispatch), so concurrent requests to one subscription
// account are serialized through the connection's max_concurrent ceiling. The
// cap is read fresh from the selected connection; a null cap (no limit) or a
// saturated queue is a no-op (fail-open). Released in the finally below.
let quotaShareConcurrencyRelease: (() => void) | null = null;
const qsConnectionId = orderedTargets[0]?.connectionId;
if (quotaShareConcurrencyEnabled && qsConnectionId) {
const qsCap = await lookupPositiveCap(qsConnectionId);
quotaShareConcurrencyRelease = await acquireQuotaShareConcurrencySlot(
orderedTargets[0],
qsCap,
{
queueTimeoutMs: config.queueTimeoutMs ?? 30000,
maxQueueSize: resolveComboQueueDepth(config),
},
log
);
}
try {
return await dispatchWithCooldownRetry();
} finally {
quotaShareConcurrencyRelease?.();
// G2: Clean up candidate registry to prevent unbounded memory growth.
_unregisterExecutionCandidates(_registeredExecutionKeys);
}

View File

@@ -24,7 +24,7 @@ import { effectiveMaxConcurrency } from "./comboPredicates.ts";
import type { ResolvedComboTarget } from "./types.ts";
/** Read a connection's positive `maxConcurrent`, or null when unset / <= 0 / on error. */
async function lookupPositiveCap(connectionId: string): Promise<number | null> {
export async function lookupPositiveCap(connectionId: string): Promise<number | null> {
try {
const conn = await getProviderConnectionById(connectionId);
const raw = (conn as { maxConcurrent?: number | null } | null)?.maxConcurrent;

View File

@@ -0,0 +1,74 @@
/**
* quotaShareConcurrency.ts — Per-connection concurrency limit for quota-share combos (FASE 2.1).
*
* The quota-share gating in selectQuotaShareTarget (combo/quotaShareStrategy.ts)
* is FAIL-OPEN: an at-cap connection is only deprioritized, never hard-blocked,
* so with a single-connection pool (the common case for a subscription account)
* concurrent requests to that account are all dispatched at once and flood it
* (→ 429 + cooldown). The quota-share dispatch path also never calls
* `semaphore.acquire` — that lived only in handleRoundRobinCombo.
*
* This helper closes that gap WITHOUT touching the fail-open selection logic:
* when the selected connection declares a positive `max_concurrent` ceiling, the
* dispatch acquires a per-CONNECTION semaphore slot keyed by connectionId (the
* cap belongs to the account, not the combo), so concurrent requests to one
* account WAIT in the queue instead of flooding it. It stays fail-open at the
* edges: no connection / no positive cap → no limit; a saturated queue / timeout
* → proceed without a slot rather than ever worsening availability.
*
* The key is connectionId-scoped (not combo-scoped like the round-robin key) so
* every quota-share request that lands on the same account shares one gate.
*/
import * as semaphore from "../rateLimitSemaphore.ts";
import type { ResolvedComboTarget } from "./types.ts";
/** Stable, connection-scoped semaphore key for the quota-share concurrency gate. */
export function quotaShareConcurrencyKey(connectionId: string): string {
return `qsconn:${connectionId}`;
}
export interface QuotaShareSlotOptions {
/** Max time a request waits in the queue before failing open (ms). */
queueTimeoutMs: number;
/** Max queued waiters before the gate fails open instead of queueing more. */
maxQueueSize: number;
}
interface SlotLogger {
warn: (tag: string, message: string) => void;
}
/**
* Acquire a per-connection concurrency slot for the selected quota-share target.
*
* Returns a release callback to invoke once the request finishes, or `null` when
* no limit applies (missing connection, `cap === null`, `cap <= 0`) or the gate
* is saturated. Fail-open is deliberate: this layer must never make a quota-share
* request fail that would otherwise have been dispatched — it only paces the ones
* it can, so a full queue / timeout proceeds WITHOUT a slot.
*/
export async function acquireQuotaShareConcurrencySlot(
target: ResolvedComboTarget | undefined,
cap: number | null,
opts: QuotaShareSlotOptions,
log: SlotLogger
): Promise<(() => void) | null> {
const connectionId = target?.connectionId ?? "";
if (!connectionId || cap === null || cap <= 0) return null;
try {
return await semaphore.acquire(quotaShareConcurrencyKey(connectionId), {
maxConcurrency: cap,
timeoutMs: opts.queueTimeoutMs,
maxQueueSize: opts.maxQueueSize,
});
} catch {
// Fail-open: a saturated queue / timeout must never worsen availability —
// proceed without a slot rather than reject a dispatchable request.
log.warn(
"COMBO",
`Quota-share concurrency: connection ${connectionId} gate saturated (cap=${cap}) — proceeding without a slot`
);
return null;
}
}

View File

@@ -43,6 +43,10 @@ type ComboCooldownWaitSettings = {
budgetMs: number;
};
type QuotaShareConcurrencyLimitSettings = {
enabled: boolean;
};
type ProviderCooldownSettings = {
enabled: boolean;
minRetryCooldownMs: number;
@@ -61,6 +65,7 @@ type ResilienceResponse = {
};
waitForCooldown: WaitForCooldownSettings;
comboCooldownWait: ComboCooldownWaitSettings;
quotaShareConcurrencyLimit: QuotaShareConcurrencyLimitSettings;
providerCooldown: ProviderCooldownSettings;
};
@@ -848,6 +853,79 @@ function ComboCooldownWaitCard({
);
}
function QuotaShareConcurrencyLimitCard({
value,
onSave,
saving,
}: {
value: QuotaShareConcurrencyLimitSettings;
onSave: (next: QuotaShareConcurrencyLimitSettings) => Promise<void>;
saving: boolean;
}) {
const t = useTranslations("settings");
const [editing, setEditing] = useState(false);
const [draft, setDraft] = useState(value);
useEffect(() => {
setDraft(value);
}, [value]);
const title =
t("resilienceQuotaShareConcurrencyTitle") || "Quota-share per-connection concurrency";
const desc =
t("resilienceQuotaShareConcurrencyDesc") ||
"For quota-share combos only: when a connection sets a Max Concurrent cap, serialize concurrent requests to that subscription account so it is never flooded past its ceiling — excess requests wait in the queue instead of getting a 429. The cap comes from each connection's Max Concurrent field; this switch only enables/disables honoring it.";
return (
<Card className="p-6">
<div className="mb-4 flex items-start justify-between gap-4">
<div className="flex items-center gap-2">
<span className="material-symbols-outlined text-xl text-primary">filter_list</span>
<h2 className="text-lg font-bold">{title}</h2>
</div>
<ActionRow
editing={editing}
saving={saving}
onEdit={() => setEditing(true)}
onCancel={() => {
setDraft(value);
setEditing(false);
}}
onSave={async () => {
await onSave(draft);
setEditing(false);
}}
/>
</div>
<p className="mb-4 text-sm text-text-muted">{desc}</p>
<div className="grid grid-cols-1 gap-3">
{editing ? (
<BooleanField
label={t("resilienceEnableServerWait") || "Enabled"}
description={
t("resilienceQuotaShareConcurrencyToggleDesc") ||
"Quota-share combos only; honors each connection's Max Concurrent cap."
}
checked={draft.enabled}
onChange={(enabled) => setDraft((prev) => ({ ...prev, enabled }))}
/>
) : (
<div className="rounded-xl border border-border bg-bg-subtle p-4">
<div className="text-xs text-text-muted">
{t("resilienceEnableServerWait") || "Enabled"}
</div>
<div className="mt-1 text-sm font-semibold text-text-main">
{value.enabled ? t("statusEnabled") : t("statusDisabled")}
</div>
</div>
)}
</div>
</Card>
);
}
function ProviderCooldownCard({
value,
onSave,
@@ -1086,6 +1164,13 @@ export default function ResilienceTab() {
saving={savingSection === "comboCooldownWait"}
onSave={(comboCooldownWait) => savePatch("comboCooldownWait", { comboCooldownWait })}
/>
<QuotaShareConcurrencyLimitCard
value={data.quotaShareConcurrencyLimit}
saving={savingSection === "quotaShareConcurrencyLimit"}
onSave={(quotaShareConcurrencyLimit) =>
savePatch("quotaShareConcurrencyLimit", { quotaShareConcurrencyLimit })
}
/>
<ProviderCooldownCard
value={data.providerCooldown}
saving={savingSection === "providerCooldown"}

View File

@@ -134,6 +134,7 @@ export async function GET() {
maxRetryWaitSec: resilience.waitForCooldown.maxRetryWaitSec,
},
comboCooldownWait: resilience.comboCooldownWait,
quotaShareConcurrencyLimit: resilience.quotaShareConcurrencyLimit,
providerCooldown: resilience.providerCooldown,
legacy: buildLegacyResilienceCompat(resilience),
});
@@ -196,6 +197,12 @@ export async function PATCH(request) {
body.comboCooldownWait as ResilienceSettingsPatch["comboCooldownWait"],
}
: {}),
...(body.quotaShareConcurrencyLimit
? {
quotaShareConcurrencyLimit:
body.quotaShareConcurrencyLimit as ResilienceSettingsPatch["quotaShareConcurrencyLimit"],
}
: {}),
...(body.providerCooldown
? {
providerCooldown: body.providerCooldown as ResilienceSettingsPatch["providerCooldown"],

View File

@@ -57,6 +57,21 @@ export interface ComboCooldownWaitSettings {
budgetMs: number;
}
/**
* Per-connection concurrency limit for quota-share (`qtSd/…`) combos (FASE 2.1).
* The quota-share gating in selectQuotaShareTarget is fail-open and cannot
* hard-limit a single-connection pool, so concurrent requests to one
* subscription account can still flood it (→ 429 + cooldown). When a connection
* declares a positive `max_concurrent` ceiling, this layer serializes concurrent
* requests to that account through a per-connection semaphore (excess requests
* wait in the queue instead of flooding). Kill-switch only: the cap itself comes
* from each connection's `max_concurrent`. Wiring lives in
* open-sse/services/combo/quotaShareConcurrency.ts.
*/
export interface QuotaShareConcurrencyLimitSettings {
enabled: boolean;
}
export interface ProviderCooldownSettings {
/**
* Minimum cooldown (ms) before a failed provider/connection can be retried.
@@ -141,6 +156,7 @@ export interface ResilienceSettings {
providerBreaker: Record<AuthCategory, ProviderBreakerProfileSettings>;
waitForCooldown: WaitForCooldownSettings;
comboCooldownWait: ComboCooldownWaitSettings;
quotaShareConcurrencyLimit: QuotaShareConcurrencyLimitSettings;
providerCooldown: ProviderCooldownSettings;
quotaPreflight: QuotaPreflightSettings;
streamRecovery: StreamRecoverySettings;
@@ -152,6 +168,7 @@ export interface ResilienceSettingsPatch {
providerBreaker?: Partial<Record<AuthCategory, Partial<ProviderBreakerProfileSettings>>>;
waitForCooldown?: Partial<WaitForCooldownSettings>;
comboCooldownWait?: Partial<ComboCooldownWaitSettings>;
quotaShareConcurrencyLimit?: Partial<QuotaShareConcurrencyLimitSettings>;
providerCooldown?: Partial<ProviderCooldownSettings>;
quotaPreflight?: Partial<QuotaPreflightSettings>;
streamRecovery?: Partial<StreamRecoverySettings>;
@@ -272,6 +289,13 @@ export const DEFAULT_RESILIENCE_SETTINGS: ResilienceSettings = {
maxAttempts: 2,
budgetMs: 8000,
},
// FASE 2.1: serialize concurrent quota-share requests per connection when the
// connection sets a max_concurrent cap, so a subscription account is not
// flooded past its concurrency ceiling. Kill-switch only (default on); the cap
// comes from each connection's max_concurrent.
quotaShareConcurrencyLimit: {
enabled: true,
},
providerCooldown: {
minRetryCooldownMs: Number(process.env.PROVIDER_COOLDOWN_MIN_MS || "5000"),
maxRetryCooldownMs: Number(process.env.PROVIDER_COOLDOWN_MAX_MS || "300000"),
@@ -550,6 +574,14 @@ function normalizeComboCooldownWaitSettings(
return { enabled, maxWaitMs, maxAttempts, budgetMs };
}
function normalizeQuotaShareConcurrencyLimitSettings(
next: unknown,
fallback: QuotaShareConcurrencyLimitSettings
): QuotaShareConcurrencyLimitSettings {
const record = asRecord(next);
return { enabled: toBoolean(record.enabled, fallback.enabled) };
}
function normalizeProviderCooldownSettings(
next: unknown,
fallback: ProviderCooldownSettings
@@ -665,6 +697,7 @@ function buildLegacyFallback(settings: JsonRecord): ResilienceSettings {
maxRetryWaitMs: waitMaxRetrySec * 1000,
},
comboCooldownWait: DEFAULT_RESILIENCE_SETTINGS.comboCooldownWait,
quotaShareConcurrencyLimit: DEFAULT_RESILIENCE_SETTINGS.quotaShareConcurrencyLimit,
providerCooldown: DEFAULT_RESILIENCE_SETTINGS.providerCooldown,
quotaPreflight: DEFAULT_RESILIENCE_SETTINGS.quotaPreflight,
streamRecovery: streamRecoveryDefaults,
@@ -708,6 +741,10 @@ export function resolveResilienceSettings(
current.comboCooldownWait,
fallback.comboCooldownWait
),
quotaShareConcurrencyLimit: normalizeQuotaShareConcurrencyLimitSettings(
current.quotaShareConcurrencyLimit,
fallback.quotaShareConcurrencyLimit
),
providerCooldown: normalizeProviderCooldownSettings(
current.providerCooldown,
fallback.providerCooldown
@@ -757,6 +794,10 @@ export function mergeResilienceSettings(
updates.comboCooldownWait,
current.comboCooldownWait
),
quotaShareConcurrencyLimit: normalizeQuotaShareConcurrencyLimitSettings(
updates.quotaShareConcurrencyLimit,
current.quotaShareConcurrencyLimit
),
providerCooldown: normalizeProviderCooldownSettings(
updates.providerCooldown,
current.providerCooldown

View File

@@ -106,6 +106,15 @@ export const comboCooldownWaitSettingsSchema = z
})
.strict();
// FASE 2.1: kill-switch for the per-connection quota-share concurrency limit.
// The cap itself comes from each connection's max_concurrent, so only `enabled`
// is configurable here.
export const quotaShareConcurrencyLimitSettingsSchema = z
.object({
enabled: z.boolean().optional(),
})
.strict();
export const providerCooldownSettingsSchema = z
.object({
enabled: z.boolean().optional(),
@@ -146,6 +155,7 @@ export const updateResilienceSchema = z
.optional(),
waitForCooldown: waitForCooldownSettingsSchema.optional(),
comboCooldownWait: comboCooldownWaitSettingsSchema.optional(),
quotaShareConcurrencyLimit: quotaShareConcurrencyLimitSettingsSchema.optional(),
providerCooldown: providerCooldownSettingsSchema.optional(),
profiles: z
.object({
@@ -164,6 +174,7 @@ export const updateResilienceSchema = z
!value.providerBreaker &&
!value.waitForCooldown &&
!value.comboCooldownWait &&
!value.quotaShareConcurrencyLimit &&
!value.providerCooldown &&
!value.profiles &&
!value.defaults

View File

@@ -0,0 +1,132 @@
/**
* tests/unit/combo/quota-share-concurrency.test.ts
*
* FASE 2.1: the per-connection concurrency slot for quota-share combos. These
* tests pin the contract of acquireQuotaShareConcurrencySlot against the real
* semaphore module: no limit when there is no cap, a stable connection-scoped
* key, genuine serialization (a second request WAITS until the first releases),
* and fail-open behavior when the queue is saturated.
*/
import test from "node:test";
import assert from "node:assert/strict";
import * as semaphore from "../../../open-sse/services/rateLimitSemaphore.ts";
import {
quotaShareConcurrencyKey,
acquireQuotaShareConcurrencySlot,
} from "../../../open-sse/services/combo/quotaShareConcurrency.ts";
const noopLog = { warn: () => {} };
function target(connectionId: string) {
return {
connectionId,
modelStr: "p/m",
executionKey: "p/m",
provider: "p",
stepId: "s",
label: "p/m",
} as never;
}
const wait = (ms: number) => new Promise((r) => setTimeout(r, ms));
test("quotaShareConcurrencyKey is stable and connection-scoped", () => {
assert.equal(quotaShareConcurrencyKey("abc"), "qsconn:abc");
assert.equal(quotaShareConcurrencyKey("abc"), quotaShareConcurrencyKey("abc"));
assert.notEqual(quotaShareConcurrencyKey("a"), quotaShareConcurrencyKey("b"));
});
test("no slot when cap is null (no per-connection limit → unchanged behavior)", async () => {
semaphore.resetAll();
const release = await acquireQuotaShareConcurrencySlot(
target("c1"),
null,
{ queueTimeoutMs: 50, maxQueueSize: 10 },
noopLog
);
assert.equal(release, null);
});
test("no slot when cap <= 0", async () => {
semaphore.resetAll();
assert.equal(
await acquireQuotaShareConcurrencySlot(
target("c1"),
0,
{ queueTimeoutMs: 50, maxQueueSize: 10 },
noopLog
),
null
);
});
test("no slot when connectionId is empty", async () => {
semaphore.resetAll();
const release = await acquireQuotaShareConcurrencySlot(
target(""),
1,
{ queueTimeoutMs: 50, maxQueueSize: 10 },
noopLog
);
assert.equal(release, null);
});
test("acquires a slot when a positive cap is set", async () => {
semaphore.resetAll();
const release = await acquireQuotaShareConcurrencySlot(
target("c1"),
1,
{ queueTimeoutMs: 50, maxQueueSize: 10 },
noopLog
);
assert.equal(typeof release, "function");
release!();
});
test("cap=1 serializes: a second concurrent request WAITS until the first releases", async () => {
semaphore.resetAll();
const opts = { queueTimeoutMs: 1000, maxQueueSize: 10 };
const r1 = await acquireQuotaShareConcurrencySlot(target("c1"), 1, opts, noopLog);
assert.equal(typeof r1, "function", "first request acquires the only slot");
let secondResolved = false;
const p2 = acquireQuotaShareConcurrencySlot(target("c1"), 1, opts, noopLog).then((r) => {
secondResolved = true;
return r;
});
await wait(60);
assert.equal(
secondResolved,
false,
"second request is still queued while the first holds the slot"
);
r1!(); // release the first
const r2 = await p2;
assert.equal(secondResolved, true, "second request resolves only after the first releases");
assert.equal(typeof r2, "function", "second request then acquires the freed slot");
r2!();
});
test("fail-open: a saturated queue proceeds without a slot (null), never blocks", async () => {
semaphore.resetAll();
const opts = { queueTimeoutMs: 1000, maxQueueSize: 0 };
const r1 = await acquireQuotaShareConcurrencySlot(target("c1"), 1, opts, noopLog);
assert.equal(typeof r1, "function", "first request acquires");
const r2 = await acquireQuotaShareConcurrencySlot(target("c1"), 1, opts, noopLog);
assert.equal(r2, null, "queue full → fail-open null (availability never worsened)");
r1!();
});
test("different connections have independent gates (no cross-contention)", async () => {
semaphore.resetAll();
const opts = { queueTimeoutMs: 50, maxQueueSize: 0 };
const r1 = await acquireQuotaShareConcurrencySlot(target("c1"), 1, opts, noopLog);
const r2 = await acquireQuotaShareConcurrencySlot(target("c2"), 1, opts, noopLog);
assert.equal(typeof r1, "function");
assert.equal(typeof r2, "function", "a different connection has its own slot");
r1!();
r2!();
});

View File

@@ -0,0 +1,66 @@
/**
* TDD — ResilienceSettings.quotaShareConcurrencyLimit (FASE 2.1, the
* per-connection concurrency kill-switch for quota-share combos). Locks the
* default, the resolve/merge round-trip, and the boolean normalization.
*/
import { test } from "node:test";
import assert from "node:assert/strict";
import {
DEFAULT_RESILIENCE_SETTINGS,
mergeResilienceSettings,
resolveResilienceSettings,
type ResilienceSettings,
} from "../../src/lib/resilience/settings.ts";
function cloneDefaults(): ResilienceSettings {
return structuredClone(DEFAULT_RESILIENCE_SETTINGS);
}
test("default quotaShareConcurrencyLimit is enabled (kill-switch on)", () => {
assert.deepEqual(cloneDefaults().quotaShareConcurrencyLimit, { enabled: true });
});
test("resolveResilienceSettings returns the default block when nothing is stored", () => {
const resolved = resolveResilienceSettings({});
assert.deepEqual(
resolved.quotaShareConcurrencyLimit,
DEFAULT_RESILIENCE_SETTINGS.quotaShareConcurrencyLimit
);
});
test("resolveResilienceSettings reads a stored disable override", () => {
const resolved = resolveResilienceSettings({
resilienceSettings: { quotaShareConcurrencyLimit: { enabled: false } },
});
assert.equal(resolved.quotaShareConcurrencyLimit.enabled, false);
});
test("mergeResilienceSettings round-trips a disable patch", () => {
const current = cloneDefaults();
const next = mergeResilienceSettings(current, {
quotaShareConcurrencyLimit: { enabled: false },
});
assert.equal(next.quotaShareConcurrencyLimit.enabled, false);
// unrelated blocks are preserved
assert.deepEqual(next.comboCooldownWait, current.comboCooldownWait);
});
test("garbage values fall back to the current enabled flag", () => {
const resolved = resolveResilienceSettings({
resilienceSettings: {
quotaShareConcurrencyLimit: { enabled: "nope" as unknown as boolean },
},
});
// non-boolean → falls back to the default (true)
assert.equal(resolved.quotaShareConcurrencyLimit.enabled, true);
});
test("an explicit re-enable patch is honored", () => {
const disabled = mergeResilienceSettings(cloneDefaults(), {
quotaShareConcurrencyLimit: { enabled: false },
});
const reenabled = mergeResilienceSettings(disabled, {
quotaShareConcurrencyLimit: { enabled: true },
});
assert.equal(reenabled.quotaShareConcurrencyLimit.enabled, true);
});