pjandro/OmniRoute
1
0
Fork 0
mirror of https://github.com/diegosouzapw/OmniRoute.git synced 2026-08-08 00:02:20 +03:00
Code Issues Packages Projects Releases Wiki Activity

docs(resilience): document Quota-Share Concurrency Control (max_concurrent + serialization + cooldown-wait) (#4980)

Browse Source 
Documents the v3.8.36 quota-share concurrency layers in RESILIENCE_GUIDE.md:
per-connection max_concurrent cap, the quota-share request serialization semaphore
(FASE 2.1, qsconn:<connectionId>, fail-open, kill-switch), and the combo
cooldown-aware retry — so operators know how to cap a subscription account's
concurrency and why the routing gate alone cannot contain a single-connection flood.

Co-authored-by: Diego Rodrigues de Sa e Souza <souzamiriamrodrigues790@gmail.com>
This commit is contained in:
Diego Rodrigues de Sa e Souza
2026-06-25 00:28:30 -03:00
committed by GitHub
parent 1f685ebcd7
commit d648e215f8
1 changed files with 40 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

40
docs/architecture/RESILIENCE_GUIDE.md
View File

@@ -159,6 +159,46 @@ lockout *state* is ephemeral.
---
## 4. Quota-Share Concurrency Control (v3.8.36)
Subscription accounts (GLM, MiniMax, etc.) often accept only ~1–3 concurrent
requests; exceeding that triggers 429s and cooldowns. This is acute under
**quota-share** (`qtSd/…`) combos, where several API keys share one upstream
account. Three layers keep a shared account from being flooded.
### Per-connection concurrency cap (`max_concurrent`)
Each provider connection can declare a `max_concurrent` ceiling
(`provider_connections.max_concurrent`, set in the connection modal / API / DB).
Leave it empty for no limit. This is the single knob that drives the serialization
layer below — set it to the account's real concurrency (e.g. GLM ~1, MiniMax ~2).
### Quota-share request serialization
When a quota-share dispatch targets a connection that declares a positive
`max_concurrent`, concurrent requests to that **account** are serialized through a
per-connection semaphore (key `qsconn:<connectionId>`): excess requests **wait in
the queue** instead of flooding the account. It is **fail-open** — a saturated
queue or timeout proceeds without a slot rather than ever rejecting a dispatchable
request. Toggle in **Settings → Resilience → Quota-share per-connection
concurrency** (`resilienceSettings.quotaShareConcurrencyLimit.enabled`, default
on). Without a `max_concurrent` cap the behavior is unchanged.
> The quota-share routing gate (`selectQuotaShareTarget`, DRR + P2C) is itself
> fail-open and only *deprioritizes* an at-cap connection — with a
> single-connection pool it cannot hard-limit, so this semaphore is what actually
> contains the flood.
### Combo cooldown-aware retry
For quota-share combos only, a request that would crystallize a 429 for a SHORT
transient cooldown waits it out and re-dispatches instead of returning the 429.
Bounded by `comboCooldownWait` (`enabled`, `maxWaitMs` 5s, `maxAttempts` 2,
`budgetMs` 8s) in **Settings → Resilience**. It never waits on `quota_exhausted`
(locked until midnight) or auth/not-found reasons.
---
## Other Resilience Features
- **14 routing strategies** (priority, weighted, round-robin, context-relay, fill-first, p2c, random, least-used, cost-optimized, reset-aware, strict-random, auto, lkgp, context-optimized) — see [AUTO-COMBO.md](../routing/AUTO-COMBO.md).