From d648e215f843c1e69445323dbb6a5904064b491c Mon Sep 17 00:00:00 2001 From: Diego Rodrigues de Sa e Souza <8016841+diegosouzapw@users.noreply.github.com> Date: Thu, 25 Jun 2026 00:28:30 -0300 Subject: [PATCH] docs(resilience): document Quota-Share Concurrency Control (max_concurrent + serialization + cooldown-wait) (#4980) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Documents the v3.8.36 quota-share concurrency layers in RESILIENCE_GUIDE.md: per-connection max_concurrent cap, the quota-share request serialization semaphore (FASE 2.1, qsconn:, fail-open, kill-switch), and the combo cooldown-aware retry — so operators know how to cap a subscription account's concurrency and why the routing gate alone cannot contain a single-connection flood. Co-authored-by: Diego Rodrigues de Sa e Souza --- docs/architecture/RESILIENCE_GUIDE.md | 40 +++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/docs/architecture/RESILIENCE_GUIDE.md b/docs/architecture/RESILIENCE_GUIDE.md index d64af0a745..9c5187b92d 100644 --- a/docs/architecture/RESILIENCE_GUIDE.md +++ b/docs/architecture/RESILIENCE_GUIDE.md @@ -159,6 +159,46 @@ lockout *state* is ephemeral. --- +## 4. Quota-Share Concurrency Control (v3.8.36) + +Subscription accounts (GLM, MiniMax, etc.) often accept only ~1–3 concurrent +requests; exceeding that triggers 429s and cooldowns. This is acute under +**quota-share** (`qtSd/…`) combos, where several API keys share one upstream +account. Three layers keep a shared account from being flooded. + +### Per-connection concurrency cap (`max_concurrent`) + +Each provider connection can declare a `max_concurrent` ceiling +(`provider_connections.max_concurrent`, set in the connection modal / API / DB). +Leave it empty for no limit. This is the single knob that drives the serialization +layer below — set it to the account's real concurrency (e.g. GLM ~1, MiniMax ~2). + +### Quota-share request serialization + +When a quota-share dispatch targets a connection that declares a positive +`max_concurrent`, concurrent requests to that **account** are serialized through a +per-connection semaphore (key `qsconn:`): excess requests **wait in +the queue** instead of flooding the account. It is **fail-open** — a saturated +queue or timeout proceeds without a slot rather than ever rejecting a dispatchable +request. Toggle in **Settings → Resilience → Quota-share per-connection +concurrency** (`resilienceSettings.quotaShareConcurrencyLimit.enabled`, default +on). Without a `max_concurrent` cap the behavior is unchanged. + +> The quota-share routing gate (`selectQuotaShareTarget`, DRR + P2C) is itself +> fail-open and only *deprioritizes* an at-cap connection — with a +> single-connection pool it cannot hard-limit, so this semaphore is what actually +> contains the flood. + +### Combo cooldown-aware retry + +For quota-share combos only, a request that would crystallize a 429 for a SHORT +transient cooldown waits it out and re-dispatches instead of returning the 429. +Bounded by `comboCooldownWait` (`enabled`, `maxWaitMs` 5s, `maxAttempts` 2, +`budgetMs` 8s) in **Settings → Resilience**. It never waits on `quota_exhausted` +(locked until midnight) or auth/not-found reasons. + +--- + ## Other Resilience Features - **14 routing strategies** (priority, weighted, round-robin, context-relay, fill-first, p2c, random, least-used, cost-optimized, reset-aware, strict-random, auto, lkgp, context-optimized) — see [AUTO-COMBO.md](../routing/AUTO-COMBO.md).