OmniRoute/docs/RESILIENCE_GUIDE.md

🛡️ Resilience Guide — OmniRoute

How OmniRoute keeps your AI coding workflow running when providers fail.

Overview

OmniRoute implements a multi-layered resilience system that ensures zero downtime:

Client Request
  → Rate Limit Check (per-IP, per-connection)
  → Combo Routing (13 strategies)
  → Connection Selection (P2C, round-robin, etc.)
  → Request Queue & Pacing
  → Execute (provider-specific executor)
    → On Failure:
      → Connection Cooldown (exponential backoff)
      → Circuit Breaker (provider-level)
      → Wait For Cooldown (auto-retry)
      → Next Combo Target (fallback chain)
  → Response

---

Request Queue & Pacing

Per-connection request buckets smooth bursts before they hit upstream rate caps.

Configure in Dashboard → Settings → Resilience:

Setting	Default	Description
Queue Size	10	Max queued requests per connection
Pacing Interval	0ms	Minimum gap between requests
Max Concurrent	5	Simultaneous requests per connection

---

Connection Cooldown

A single connection cools down after retryable failures. Features:

• Exponential Backoff — progressively longer cooldowns after each failure
• Retry-After Header Support — respects upstream hints
• Configurable Base/Max — tune cooldown duration per use case
• Auto-Recovery — connection automatically becomes available after cooldown expires

---

Circuit Breaker

Provider-level protection against cascading failures:

1. Connection-scoped 429 rate limits stay in Connection Cooldown (don't trip the breaker)
2. Provider-wide transient errors (5xx, network timeouts) increment the failure counter
3. Breaker trips only after fallback is exhausted AND the provider still fails
4. Recovery — breaker automatically moves to half-open state after timeout, tests with probe request

Configure thresholds in Dashboard → Settings → Resilience.

---

Wait For Cooldown

Instead of immediately failing when all connections are in cooldown, OmniRoute can wait for the earliest connection to expire and retry:

• Automatic — server waits for the earliest cooldown to expire
• Transparent — client sees a slightly delayed response instead of an error
• Configurable — enable/disable per combo or globally

---

Anti-Thundering Herd

When multiple concurrent requests hit a failing provider simultaneously:

• Mutex Protection — only one retry attempt at a time per connection
• Semaphore — limits concurrent retry storms across connections
• Deduplication — identical requests within 5s window are deduplicated

---

Combo Fallback Chains

The primary resilience mechanism. Configure in Dashboard → Combos:

Combo: "always-on"
  1. cc/claude-opus-4-7        ← Primary (subscription)
  2. cx/gpt-5.2-codex          ← Secondary (subscription)
  3. glm/glm-4.7               ← Cheap backup ($0.5/1M)
  4. if/kimi-k2-thinking        ← Free fallback (unlimited)

When provider #1 fails (quota, rate, or health), OmniRoute automatically routes to #2, then #3, then #4 — with zero manual intervention.

13 Routing Strategies

Strategy	Description
priority	First available in order
weighted	Weighted distribution
fill-first	Fill primary before moving
round-robin	Rotate through all targets
p2c	Power-of-two choices (quota-aware)
random	Random selection
least-used	Least recently used
cost-optimized	Cheapest available
strict-random	True random (no tracking)
auto	OmniRoute selects based on context
lkgp	Last Known Good Provider
context-optimized	Best for current context window
context-relay	Session handoff during rotation

---

TLS Fingerprint Spoofing

OmniRoute makes proxied traffic look like legitimate browser/CLI requests:

• Browser-like TLS via wreq-js — prevents bot detection
• CLI Fingerprint Matching — reorders headers and body fields to match native CLI binary signatures (Claude Code, Codex, etc.)
• Proxy IP Preservation — stealth features work on top of proxy IP masking

---

Health Dashboard

Monitor all resilience components in real-time at Dashboard → Health:

• Uptime — server uptime and last restart
• Provider Breaker States — open/closed/half-open per provider
• Connection Cooldowns — active cooldowns with expiry times
• Cache Stats — signature + semantic cache hit rates
• Lockouts — API key lockouts and IP bans
• Latency — p50/p95/p99 percentiles

---

See Also

• Architecture Guide — System architecture and internals
• User Guide — Providers, combos, CLI integration
• Auto-Combo Engine — 6-factor scoring, mode packs