diff --git a/docs/architecture/RESILIENCE_GUIDE.md b/docs/architecture/RESILIENCE_GUIDE.md index d64af0a745..9c5187b92d 100644 --- a/docs/architecture/RESILIENCE_GUIDE.md +++ b/docs/architecture/RESILIENCE_GUIDE.md @@ -159,6 +159,46 @@ lockout *state* is ephemeral. --- +## 4. Quota-Share Concurrency Control (v3.8.36) + +Subscription accounts (GLM, MiniMax, etc.) often accept only ~1–3 concurrent +requests; exceeding that triggers 429s and cooldowns. This is acute under +**quota-share** (`qtSd/…`) combos, where several API keys share one upstream +account. Three layers keep a shared account from being flooded. + +### Per-connection concurrency cap (`max_concurrent`) + +Each provider connection can declare a `max_concurrent` ceiling +(`provider_connections.max_concurrent`, set in the connection modal / API / DB). +Leave it empty for no limit. This is the single knob that drives the serialization +layer below — set it to the account's real concurrency (e.g. GLM ~1, MiniMax ~2). + +### Quota-share request serialization + +When a quota-share dispatch targets a connection that declares a positive +`max_concurrent`, concurrent requests to that **account** are serialized through a +per-connection semaphore (key `qsconn:`): excess requests **wait in +the queue** instead of flooding the account. It is **fail-open** — a saturated +queue or timeout proceeds without a slot rather than ever rejecting a dispatchable +request. Toggle in **Settings → Resilience → Quota-share per-connection +concurrency** (`resilienceSettings.quotaShareConcurrencyLimit.enabled`, default +on). Without a `max_concurrent` cap the behavior is unchanged. + +> The quota-share routing gate (`selectQuotaShareTarget`, DRR + P2C) is itself +> fail-open and only *deprioritizes* an at-cap connection — with a +> single-connection pool it cannot hard-limit, so this semaphore is what actually +> contains the flood. + +### Combo cooldown-aware retry + +For quota-share combos only, a request that would crystallize a 429 for a SHORT +transient cooldown waits it out and re-dispatches instead of returning the 429. +Bounded by `comboCooldownWait` (`enabled`, `maxWaitMs` 5s, `maxAttempts` 2, +`budgetMs` 8s) in **Settings → Resilience**. It never waits on `quota_exhausted` +(locked until midnight) or auth/not-found reasons. + +--- + ## Other Resilience Features - **14 routing strategies** (priority, weighted, round-robin, context-relay, fill-first, p2c, random, least-used, cost-optimized, reset-aware, strict-random, auto, lkgp, context-optimized) — see [AUTO-COMBO.md](../routing/AUTO-COMBO.md).