mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-07-26 09:52:11 +03:00
- Add collapsible '16 Real Pain Points' section to all 30 READMEs - Fix 5 files bypassing dataPaths.ts with hardcoded os.homedir() (closes #156) - Add per-provider User-Agent env var overrides in base executor (closes #155) - Sync .env and .env.example with 9 provider UA defaults - Update CHANGELOG.md for v1.7.0
This commit is contained in:
diegosouzapw
259
README.md
259
README.md
@@ -169,6 +169,265 @@ Result: Never stop coding, minimal cost
|
||||
|
||||
---
|
||||
|
||||
## 🎯 What OmniRoute Solves — 16 Real Pain Points
|
||||
|
||||
> **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to zero observability.
|
||||
|
||||
<details>
|
||||
<summary><b>💸 1. "I pay for an expensive subscription but still get interrupted by limits"</b></summary>
|
||||
|
||||
Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Even paying, quota has a ceiling — 5h of usage, weekly limits, or per-minute rate limits. Mid-coding session, the provider stops responding and the developer loses flow and productivity.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
|
||||
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
|
||||
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
|
||||
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
|
||||
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔌 2. "I need to use multiple providers but each has a different API"</b></summary>
|
||||
|
||||
OpenAI uses one format, Claude (Anthropic) uses another, Gemini yet another. If a dev wants to test models from different providers or fallback between them, they need to reconfigure SDKs, change endpoints, deal with incompatible formats. Custom providers (FriendLI, NIM) have non-standard model endpoints.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Endpoint** — A single `http://localhost:20128/v1` serves as proxy for all 36+ providers
|
||||
- **Format Translation** — Automatic and transparent: OpenAI ↔ Claude ↔ Gemini ↔ Responses API
|
||||
- **Response Sanitization** — Strips non-standard fields (`x_groq`, `usage_breakdown`, `service_tier`) that break OpenAI SDK v1.83+
|
||||
- **Role Normalization** — Converts `developer` → `system` for non-OpenAI providers; `system` → `user` for GLM/ERNIE
|
||||
- **Think Tag Extraction** — Extracts `<think>` blocks from models like DeepSeek R1 into standardized `reasoning_content`
|
||||
- **Structured Output for Gemini** — `json_schema` → `responseMimeType`/`responseSchema` automatic conversion
|
||||
- **`stream` defaults to `false`** — Aligns with OpenAI spec, avoiding unexpected SSE in Python/Rust/Go SDKs
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌐 3. "My AI provider blocks my region/country"</b></summary>
|
||||
|
||||
Providers like OpenAI/Codex block access from certain geographic regions. Users get errors like `unsupported_country_region_territory` during OAuth and API connections. This is especially frustrating for developers from developing countries.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **3-Level Proxy Config** — Configurable proxy at 3 levels: global (all traffic), per-provider (one provider only), and per-connection/key
|
||||
- **Color-Coded Proxy Badges** — Visual indicators: 🟢 global proxy, 🟡 provider proxy, 🔵 connection proxy, always showing the IP
|
||||
- **OAuth Token Exchange Through Proxy** — OAuth flow also goes through the proxy, solving `unsupported_country_region_territory`
|
||||
- **Connection Tests via Proxy** — Connection tests use the configured proxy (no more direct bypass)
|
||||
- **SOCKS5 Support** — Full SOCKS5 proxy support for outbound routing
|
||||
- **TLS Fingerprint Spoofing** — Browser-like TLS fingerprint via `wreq-js` to bypass bot detection
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🆓 4. "I want to use AI for coding but I have no money"</b></summary>
|
||||
|
||||
Not everyone can pay $20–200/month for AI subscriptions. Students, devs from emerging countries, hobbyists, and freelancers need access to quality models at zero cost.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Free Tier Providers Built-in** — Native support for 100% free providers: iFlow (8 unlimited models), Qwen (3 unlimited models), Kiro (Claude for free), Gemini CLI (180K/month free)
|
||||
- **Free-Only Combos** — Chain `gc/gemini-3-flash → if/kimi-k2-thinking → qw/qwen3-coder-plus` = $0/month with zero downtime
|
||||
- **NVIDIA NIM Free Credits** — 1000 free credits integrated
|
||||
- **Cost Optimized Strategy** — Routing strategy that automatically chooses the cheapest available provider
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔒 5. "I need to protect my AI gateway from unauthorized access"</b></summary>
|
||||
|
||||
When exposing an AI gateway to the network (LAN, VPS, Docker), anyone with the address can consume the developer's tokens/quota. Without protection, APIs are vulnerable to misuse, prompt injection, and abuse.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **API Key Management** — Generation, rotation, and scoping per provider with a dedicated `/dashboard/api-manager` page
|
||||
- **Model-Level Permissions** — Restrict API keys to specific models (`openai/*`, wildcard patterns), with Allow All/Restrict toggle
|
||||
- **API Endpoint Protection** — Require a key for `/v1/models` and block specific providers from the listing
|
||||
- **Auth Guard + CSRF Protection** — All dashboard routes protected with `withAuth` middleware + CSRF tokens
|
||||
- **Rate Limiter** — Per-IP rate limiting with configurable windows
|
||||
- **IP Filtering** — Allowlist/blocklist for access control
|
||||
- **Prompt Injection Guard** — Sanitization against malicious prompt patterns
|
||||
- **AES-256-GCM Encryption** — Credentials encrypted at rest
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🛑 6. "My provider went down and I lost my coding flow"</b></summary>
|
||||
|
||||
AI providers can become unstable, return 5xx errors, or hit temporary rate limits. If a dev depends on a single provider, they're interrupted. Without circuit breakers, repeated retries can crash the application.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Circuit Breaker per-provider** — Auto-open/close with configurable thresholds and cooldown (Closed/Open/Half-Open)
|
||||
- **Exponential Backoff** — Progressive retry delays
|
||||
- **Anti-Thundering Herd** — Mutex + semaphore protection against concurrent retry storms
|
||||
- **Combo Fallback Chains** — If the primary provider fails, automatically falls through the chain with no intervention
|
||||
- **Combo Circuit Breaker** — Auto-disables failing providers within a combo chain
|
||||
- **Health Dashboard** — Uptime monitoring, circuit breaker states, lockouts, cache stats, p50/p95/p99 latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔧 7. "Configuring each AI tool is tedious and repetitive"</b></summary>
|
||||
|
||||
Developers use Cursor, Claude Code, Codex CLI, OpenClaw, Gemini CLI, Kilo Code... Each tool needs a different config (API endpoint, key, model). Reconfiguring when switching providers or models is a waste of time.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **CLI Tools Dashboard** — Dedicated page with one-click setup for Claude Code, Codex CLI, OpenClaw, Kilo Code, Antigravity, Cline
|
||||
- **GitHub Copilot Config Generator** — Generates `chatLanguageModels.json` for VS Code with bulk model selection
|
||||
- **Onboarding Wizard** — Guided 4-step setup for first-time users
|
||||
- **One endpoint, all models** — Configure `http://localhost:20128/v1` once, access 36+ providers
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔑 8. "Managing OAuth tokens from multiple providers is hell"</b></summary>
|
||||
|
||||
Claude Code, Codex, Gemini CLI, Copilot — all use OAuth 2.0 with expiring tokens. Developers need to re-authenticate constantly, deal with `client_secret is missing`, `redirect_uri_mismatch`, and failures on remote servers. OAuth on LAN/VPS is particularly problematic.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Auto Token Refresh** — OAuth tokens refresh in background before expiration
|
||||
- **OAuth 2.0 (PKCE) Built-in** — Automatic flow for Claude Code, Codex, Gemini CLI, Copilot, Kiro, Qwen, iFlow
|
||||
- **Multi-Account OAuth** — Multiple accounts per provider via JWT/ID token extraction
|
||||
- **OAuth LAN/Remote Fix** — Private IP detection for `redirect_uri` + manual URL mode for remote servers
|
||||
- **OAuth Behind Nginx** — Uses `window.location.origin` for reverse proxy compatibility
|
||||
- **Remote OAuth Guide** — Step-by-step guide for Google Cloud credentials on VPS/Docker
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📊 9. "I don't know how much I'm spending or where"</b></summary>
|
||||
|
||||
Developers use multiple paid providers but have no unified view of spending. Each provider has its own billing dashboard, but there's no consolidated view. Unexpected costs can pile up.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Cost Analytics Dashboard** — Per-token cost tracking and budget management per provider
|
||||
- **Budget Limits per Tier** — Spending ceiling per tier that triggers automatic fallback
|
||||
- **Per-Model Pricing Configuration** — Configurable prices per model
|
||||
- **Usage Statistics Per API Key** — Request count and last-used timestamp per key
|
||||
- **Analytics Dashboard** — Stat cards, model usage chart, provider table with success rates and latency
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🐛 10. "I can't diagnose errors and problems in AI calls"</b></summary>
|
||||
|
||||
When a call fails, the dev doesn't know if it was a rate limit, expired token, wrong format, or provider error. Fragmented logs across different terminals. Without observability, debugging is trial-and-error.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Unified Logs Dashboard** — 4 tabs: Request Logs, Proxy Logs, Audit Logs, Console
|
||||
- **Console Log Viewer** — Real-time terminal-style viewer with color-coded levels, auto-scroll, search, filter
|
||||
- **SQLite Proxy Logs** — Persistent logs that survive server restarts
|
||||
- **Translator Playground** — 4 debugging modes: Playground (format translation), Chat Tester (round-trip), Test Bench (batch), Live Monitor (real-time)
|
||||
- **Request Telemetry** — p50/p95/p99 latency + X-Request-Id tracing
|
||||
- **File-Based Logging with Rotation** — Console interceptor captures everything to JSON log with size-based rotation
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🏗️ 11. "Deploying and maintaining the gateway is complex"</b></summary>
|
||||
|
||||
Installing, configuring, and maintaining an AI proxy across different environments (local, VPS, Docker, cloud) is labor-intensive. Problems like hardcoded paths, `EACCES` on directories, port conflicts, and cross-platform builds add friction.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **npm global install** — `npm install -g omniroute && omniroute` — done
|
||||
- **Docker Multi-Platform** — AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi)
|
||||
- **Docker Compose Profiles** — `base` (no CLI tools) and `cli` (with Claude Code, Codex, OpenClaw)
|
||||
- **Electron Desktop App** — Native app for Windows/macOS/Linux with system tray, auto-start, offline mode
|
||||
- **Split-Port Mode** — API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking)
|
||||
- **Cloud Sync** — Config synchronization across devices via Cloudflare Workers
|
||||
- **DB Backups** — Automatic backup, restore, export and import of all settings
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🌍 12. "The interface is English-only and my team doesn't speak English"</b></summary>
|
||||
|
||||
Teams in non-English-speaking countries, especially in Latin America, Asia, and Europe, struggle with English-only interfaces. Language barriers reduce adoption and increase configuration errors.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Dashboard i18n — 30 Languages** — All 500+ keys translated including Arabic, Bulgarian, Danish, German, Spanish, Finnish, French, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese (PT/BR), Romanian, Russian, Slovak, Swedish, Thai, Ukrainian, Vietnamese, Chinese, Filipino, English
|
||||
- **RTL Support** — Right-to-left support for Arabic and Hebrew
|
||||
- **Multi-Language READMEs** — 30 complete documentation translations
|
||||
- **Language Selector** — Globe icon in header for real-time switching
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🔄 13. "I need more than chat — I need embeddings, images, audio"</b></summary>
|
||||
|
||||
AI isn't just chat completion. Devs need to generate images, transcribe audio, create embeddings for RAG, rerank documents, and moderate content. Each API has a different endpoint and format.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Embeddings** — `/v1/embeddings` with 6 providers and 9+ models
|
||||
- **Image Generation** — `/v1/images/generations` with 4 providers and 9+ models
|
||||
- **Audio Transcription** — `/v1/audio/transcriptions` — Whisper-compatible
|
||||
- **Text-to-Speech** — `/v1/audio/speech` — Multi-provider audio synthesis
|
||||
- **Moderations** — `/v1/moderations` — Content safety checks
|
||||
- **Reranking** — `/v1/rerank` — Document relevance reranking
|
||||
- **Responses API** — Full `/v1/responses` support for Codex
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🧪 14. "I have no way to test and compare quality across models"</b></summary>
|
||||
|
||||
Developers want to know which model is best for their use case — code, translation, reasoning — but comparing manually is slow. No integrated eval tools exist.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **LLM Evaluations** — Golden set testing with 10 pre-loaded cases covering greetings, math, geography, code generation, JSON compliance, translation, markdown, safety refusal
|
||||
- **4 Match Strategies** — `exact`, `contains`, `regex`, `custom` (JS function)
|
||||
- **Translator Playground Test Bench** — Batch testing with multiple inputs and expected outputs, cross-provider comparison
|
||||
- **Chat Tester** — Full round-trip with visual response rendering
|
||||
- **Live Monitor** — Real-time stream of all requests flowing through the proxy
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>📈 15. "I need to scale without losing performance"</b></summary>
|
||||
|
||||
As request volume grows, without caching the same questions generate duplicate costs. Without idempotency, duplicate requests waste processing. Per-provider rate limits must be respected.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **Semantic Cache** — Two-tier cache (signature + semantic) reduces cost and latency
|
||||
- **Request Idempotency** — 5s deduplication window for identical requests
|
||||
- **Rate Limit Detection** — Per-provider RPM, min gap, and max concurrent tracking
|
||||
- **Editable Rate Limits** — Configurable defaults in Settings → Resilience with persistence
|
||||
- **API Key Validation Cache** — 3-tier cache for production performance
|
||||
- **Health Dashboard with Telemetry** — p50/p95/p99 latency, cache stats, uptime
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>🤖 16. "I want to control model behavior globally"</b></summary>
|
||||
|
||||
Developers who want all responses in a specific language, with a specific tone, or want to limit reasoning tokens. Configuring this in every tool/request is impractical.
|
||||
|
||||
**How OmniRoute solves it:**
|
||||
|
||||
- **System Prompt Injection** — Global prompt applied to all requests
|
||||
- **Thinking Budget Validation** — Reasoning token allocation control per request (passthrough, auto, custom, adaptive)
|
||||
- **6 Routing Strategies** — Global strategies that determine how requests are distributed
|
||||
- **Wildcard Router** — `provider/*` patterns route dynamically to any provider
|
||||
- **Combo Enable/Disable Toggle** — Toggle combos directly from the dashboard
|
||||
- **Provider Toggle** — Enable/disable all connections for a provider with one click
|
||||
- **Blocked Providers** — Exclude specific providers from `/v1/models` listing
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## ⚡ Quick Start
|
||||
|
||||
**1. Install globally:**
|
||||
|
||||
Reference in New Issue
Block a user