Address bot review feedback: use .finally() instead of .then()/.catch()
so limiters.delete() runs regardless of whether stop() succeeds or
throws (e.g. already stopped by concurrent 429).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Multiple concurrent requests can receive 429 simultaneously, causing
stop() to be called on an already-stopped limiter. Add .catch() to
prevent unhandled rejection.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When a provider returns 429 (rate limit exceeded), the rate limit manager
was setting reservoir=0 and waiting for reservoirRefreshInterval before
releasing queued requests. For providers with long rate limit windows
(e.g. Codex with hours-long resets), this caused all queued requests to
hang indefinitely — they never timed out or returned an error.
This prevented upstream callers (e.g. LiteLLM) from triggering fallback
to alternative providers, effectively making the entire model unavailable
until the rate limit window expired.
Fix: on 429, call limiter.stop({ dropWaitingJobs: true }) to immediately
fail all queued requests, then delete the limiter from the Map so
getLimiter() creates a fresh instance for subsequent requests.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When all provider quotas are exhausted (reservoir=0 after repeated 429s),
Bottleneck's schedule() would queue requests indefinitely since no maxWait
was configured. Clients (Cursor, Claude Code, VS Code) would hang forever.
Fix: add maxWait=120000 (2min, configurable via RATE_LIMIT_MAX_WAIT_MS env)
to DEFAULT_SETTINGS and all three Bottleneck constructors. When a job waits
longer than maxWait, Bottleneck rejects with a BottleneckError which
propagates as a 502/503 error to the client — a clean fail-fast instead
of infinite hang.
- Issue #199: proxy_ prefix on tool names is now automatically disabled
when routing to non-Claude backends (Gemini, Compatible, etc.).
Prevents tool name mismatches in OpenCode, Cursor, and other clients.
- Issue #198: Added customRpm/customTpm support per provider connection.
Users can configure custom rate limits via connection settings,
overriding the default auto-learned limits from response headers.
OmniRoute is an intelligent API gateway that unifies 20+ AI providers behind a single
OpenAI-compatible endpoint. Features include intelligent routing with 6 strategies,
multi-format translation (OpenAI/Claude/Gemini/Responses API), circuit breakers,
semantic caching, combo fallback chains, real-time health monitoring, and a full
dashboard with provider management, analytics, and CLI tool integration.
Key highlights:
- 20+ providers (Claude Code, Codex, Gemini CLI, GitHub Copilot, iFlow, Qwen, Kiro, etc.)
- 6 routing strategies (Fill First, Round Robin, P2C, Random, Least Used, Cost Optimized)
- Export/Import database backup with full archive support
- Translator Playground with 4 modes (Playground, Chat Tester, Test Bench, Live Monitor)
- 100% TypeScript across src/ and open-sse/
- Docker support with multi-stage builds
- Comprehensive documentation and 9 dashboard screenshots
