mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-07-31 04:12:10 +03:00
29c170f83a
Break out long-form README content into focused documentation pages for core onboarding and operations topics. Refresh the README navigation to point readers to the new guides while keeping high-level product sections easier to scan.
146 lines
5.2 KiB
Markdown
146 lines
5.2 KiB
Markdown
# 🛡️ Resilience Guide — OmniRoute
|
|
|
|
> How OmniRoute keeps your AI coding workflow running when providers fail.
|
|
|
|
## Overview
|
|
|
|
OmniRoute implements a multi-layered resilience system that ensures zero downtime:
|
|
|
|
```
|
|
Client Request
|
|
→ Rate Limit Check (per-IP, per-connection)
|
|
→ Combo Routing (13 strategies)
|
|
→ Connection Selection (P2C, round-robin, etc.)
|
|
→ Request Queue & Pacing
|
|
→ Execute (provider-specific executor)
|
|
→ On Failure:
|
|
→ Connection Cooldown (exponential backoff)
|
|
→ Circuit Breaker (provider-level)
|
|
→ Wait For Cooldown (auto-retry)
|
|
→ Next Combo Target (fallback chain)
|
|
→ Response
|
|
```
|
|
|
|
---
|
|
|
|
## Request Queue & Pacing
|
|
|
|
Per-connection request buckets smooth bursts before they hit upstream rate caps.
|
|
|
|
Configure in `Dashboard → Settings → Resilience`:
|
|
|
|
| Setting | Default | Description |
|
|
| --------------- | ------- | ------------------------------------ |
|
|
| Queue Size | `10` | Max queued requests per connection |
|
|
| Pacing Interval | `0ms` | Minimum gap between requests |
|
|
| Max Concurrent | `5` | Simultaneous requests per connection |
|
|
|
|
---
|
|
|
|
## Connection Cooldown
|
|
|
|
A single connection cools down after retryable failures. Features:
|
|
|
|
- **Exponential Backoff** — progressively longer cooldowns after each failure
|
|
- **`Retry-After` Header Support** — respects upstream hints
|
|
- **Configurable Base/Max** — tune cooldown duration per use case
|
|
- **Auto-Recovery** — connection automatically becomes available after cooldown expires
|
|
|
|
---
|
|
|
|
## Circuit Breaker
|
|
|
|
Provider-level protection against cascading failures:
|
|
|
|
1. **Connection-scoped `429` rate limits** stay in Connection Cooldown (don't trip the breaker)
|
|
2. **Provider-wide transient errors** (5xx, network timeouts) increment the failure counter
|
|
3. **Breaker trips** only after fallback is exhausted AND the provider still fails
|
|
4. **Recovery** — breaker automatically moves to half-open state after timeout, tests with probe request
|
|
|
|
Configure thresholds in `Dashboard → Settings → Resilience`.
|
|
|
|
---
|
|
|
|
## Wait For Cooldown
|
|
|
|
Instead of immediately failing when all connections are in cooldown, OmniRoute can wait for the earliest connection to expire and retry:
|
|
|
|
- **Automatic** — server waits for the earliest cooldown to expire
|
|
- **Transparent** — client sees a slightly delayed response instead of an error
|
|
- **Configurable** — enable/disable per combo or globally
|
|
|
|
---
|
|
|
|
## Anti-Thundering Herd
|
|
|
|
When multiple concurrent requests hit a failing provider simultaneously:
|
|
|
|
- **Mutex Protection** — only one retry attempt at a time per connection
|
|
- **Semaphore** — limits concurrent retry storms across connections
|
|
- **Deduplication** — identical requests within 5s window are deduplicated
|
|
|
|
---
|
|
|
|
## Combo Fallback Chains
|
|
|
|
The primary resilience mechanism. Configure in `Dashboard → Combos`:
|
|
|
|
```txt
|
|
Combo: "always-on"
|
|
1. cc/claude-opus-4-7 ← Primary (subscription)
|
|
2. cx/gpt-5.2-codex ← Secondary (subscription)
|
|
3. glm/glm-4.7 ← Cheap backup ($0.5/1M)
|
|
4. if/kimi-k2-thinking ← Free fallback (unlimited)
|
|
```
|
|
|
|
When provider #1 fails (quota, rate, or health), OmniRoute automatically routes to #2, then #3, then #4 — with zero manual intervention.
|
|
|
|
### 13 Routing Strategies
|
|
|
|
| Strategy | Description |
|
|
| ------------------- | ---------------------------------- |
|
|
| `priority` | First available in order |
|
|
| `weighted` | Weighted distribution |
|
|
| `fill-first` | Fill primary before moving |
|
|
| `round-robin` | Rotate through all targets |
|
|
| `p2c` | Power-of-two choices (quota-aware) |
|
|
| `random` | Random selection |
|
|
| `least-used` | Least recently used |
|
|
| `cost-optimized` | Cheapest available |
|
|
| `strict-random` | True random (no tracking) |
|
|
| `auto` | OmniRoute selects based on context |
|
|
| `lkgp` | Last Known Good Provider |
|
|
| `context-optimized` | Best for current context window |
|
|
| `context-relay` | Session handoff during rotation |
|
|
|
|
---
|
|
|
|
## TLS Fingerprint Spoofing
|
|
|
|
OmniRoute makes proxied traffic look like legitimate browser/CLI requests:
|
|
|
|
- **Browser-like TLS** via `wreq-js` — prevents bot detection
|
|
- **CLI Fingerprint Matching** — reorders headers and body fields to match native CLI binary signatures (Claude Code, Codex, etc.)
|
|
- **Proxy IP Preservation** — stealth features work on top of proxy IP masking
|
|
|
|
---
|
|
|
|
## Health Dashboard
|
|
|
|
Monitor all resilience components in real-time at `Dashboard → Health`:
|
|
|
|
- **Uptime** — server uptime and last restart
|
|
- **Provider Breaker States** — open/closed/half-open per provider
|
|
- **Connection Cooldowns** — active cooldowns with expiry times
|
|
- **Cache Stats** — signature + semantic cache hit rates
|
|
- **Lockouts** — API key lockouts and IP bans
|
|
- **Latency** — p50/p95/p99 percentiles
|
|
|
|
---
|
|
|
|
## See Also
|
|
|
|
- [Architecture Guide](ARCHITECTURE.md) — System architecture and internals
|
|
- [User Guide](USER_GUIDE.md) — Providers, combos, CLI integration
|
|
- [Auto-Combo Engine](AUTO-COMBO.md) — 6-factor scoring, mode packs
|