mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-13 02:32:18 +03:00
Compare commits
5 Commits
chore/repo
...
docs/refre
| Author | SHA1 | Date | |
|---|---|---|---|
|
092669d692 | ||
|
|
eff133ba91 | ||
|
|
c244cd14a4 | ||
|
|
0bc823ce05 | ||
|
|
3f1402d558 |
14
AGENTS.md
14
AGENTS.md
@@ -46,7 +46,7 @@ Repository map and Reference Documentation sections below.
|
||||
|
||||
## Project at a Glance
|
||||
|
||||
**OmniRoute** — unified AI proxy/router. One endpoint, 291 LLM providers, auto-fallback.
|
||||
**OmniRoute** — unified AI proxy/router. One endpoint, 338 LLM providers, auto-fallback.
|
||||
|
||||
| Layer | Location | Purpose |
|
||||
| ------------- | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
@@ -56,9 +56,9 @@ Repository map and Reference Documentation sections below.
|
||||
| Translators | `open-sse/translator/` | Format conversion (OpenAI↔Claude↔Gemini) |
|
||||
| Transformer | `open-sse/transformer/` | Responses API ↔ Chat Completions |
|
||||
| Services | `open-sse/services/` | Combo routing, rate limits, caching, etc |
|
||||
| Database | `src/lib/db/` | SQLite domain modules (130 migrations) |
|
||||
| Database | `src/lib/db/` | SQLite domain modules (144 migrations) |
|
||||
| Domain/Policy | `src/domain/` | Policy engine, cost rules, fallback logic |
|
||||
| MCP Server | `open-sse/mcp-server/` | 105 tools (42 base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules), 3 transports (stdio / SSE / Streamable HTTP), 31 scopes |
|
||||
| MCP Server | `open-sse/mcp-server/` | 105 tools (43 base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules), 3 transports (stdio / SSE / Streamable HTTP), 31 scopes |
|
||||
| A2A Server | `src/lib/a2a/` | JSON-RPC 2.0 agent protocol |
|
||||
| Skills | `src/lib/skills/` | Extensible skill framework |
|
||||
| Memory | `src/lib/memory/` | Persistent conversational memory |
|
||||
@@ -83,7 +83,7 @@ Client → /v1/chat/completions (Next.js route)
|
||||
|
||||
API routes follow a consistent pattern: `Route → CORS preflight → Zod body validation → Optional auth (extractApiKey/isValidApiKey) → API key policy enforcement → Handler delegation (open-sse)`. No global Next.js middleware — interception is route-specific.
|
||||
|
||||
**Combo routing** (`open-sse/services/combo.ts`): 19 public strategies (priority, weighted, fill-first, round-robin, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline). Each target calls `handleSingleModel()` which wraps `handleChatCore()` with per-target error handling and circuit breaker checks. The `fusion` strategy is the exception: it fans out to a panel of models in parallel, then a judge model synthesizes one final answer (`open-sse/services/fusion.ts`). See `docs/routing/AUTO-COMBO.md` for the 13-factor Auto-Combo scoring + the full strategy table and `docs/architecture/RESILIENCE_GUIDE.md` for the 3 resilience layers.
|
||||
**Combo routing** (`open-sse/services/combo.ts`): 19 public strategies (priority, weighted, fill-first, round-robin, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline). Each target calls `handleSingleModel()` which wraps `handleChatCore()` with per-target error handling and circuit breaker checks. The `fusion` strategy is the exception: it fans out to a panel of models in parallel, then a judge model synthesizes one final answer (`open-sse/services/fusion.ts`). See `docs/routing/AUTO-COMBO.md` for the 14-factor Auto-Combo scoring + the full strategy table and `docs/architecture/RESILIENCE_GUIDE.md` for the 3 resilience layers.
|
||||
|
||||
---
|
||||
|
||||
@@ -395,7 +395,7 @@ For any non-trivial change, read the matching deep-dive first:
|
||||
| Repo navigation | `docs/architecture/REPOSITORY_MAP.md` |
|
||||
| Architecture | `docs/architecture/ARCHITECTURE.md` |
|
||||
| Engineering reference | `docs/architecture/CODEBASE_DOCUMENTATION.md` |
|
||||
| Auto-Combo (13-factor scoring, 19 strategies) | `docs/routing/AUTO-COMBO.md` |
|
||||
| Auto-Combo (14-factor scoring, 19 strategies) | `docs/routing/AUTO-COMBO.md` |
|
||||
| Resilience (3 mechanisms) | `docs/architecture/RESILIENCE_GUIDE.md` |
|
||||
| Reasoning replay | `docs/routing/REASONING_REPLAY.md` |
|
||||
| Skills framework | `docs/frameworks/SKILLS.md` |
|
||||
@@ -419,7 +419,7 @@ For any non-trivial change, read the matching deep-dive first:
|
||||
| Electron desktop app | `docs/guides/ELECTRON_GUIDE.md` |
|
||||
| Release flow | `docs/ops/RELEASE_CHECKLIST.md` |
|
||||
| Embedded services | `docs/frameworks/EMBEDDED-SERVICES.md` |
|
||||
| Quality gates (~48 scripts, allowlist policy) | `docs/architecture/QUALITY_GATES.md` |
|
||||
| Quality gates (~80 scripts, allowlist policy) | `docs/architecture/QUALITY_GATES.md` |
|
||||
|
||||
---
|
||||
|
||||
@@ -611,7 +611,7 @@ focused checks, and use a Conventional Commit message (for example, `docs: slim
|
||||
|
||||
## Quality Gates & Ratchets
|
||||
|
||||
OmniRoute has **~48 quality-gate scripts** (`scripts/check/` + `scripts/quality/`) wired
|
||||
OmniRoute has **~80 quality-gate scripts** (`scripts/check/` + `scripts/quality/`) wired
|
||||
across **9 gate-running jobs** in `.github/workflows/ci.yml` (`lint`, `quality-gate`,
|
||||
`quality-extended`, `docs-sync-strict`, `i18n-ui-coverage`, `i18n`, `pr-test-policy`,
|
||||
`test-vitest`, `sonarqube`), plus the `quality.yml` fast-gates job (PR→`release/**`) and
|
||||
|
||||
41
README.md
41
README.md
@@ -7,19 +7,19 @@
|
||||
|
||||
# 🚀 OmniRoute — The Free AI Gateway
|
||||
|
||||
<img src="./docs/diagrams/readme-hero.svg" width="100%" alt="OmniRoute — Never stop coding. Every AI tool → 291 providers — 90+ free — through one endpoint. Claude Code, Codex, Cursor, Cline, Copilot & Antigravity into FREE Claude / GPT / Gemini with auto-fallback. RTK + Caveman stacked compression saves 15–95% tokens (~89% avg) — never hit limits. 291 AI providers · 90+ free tiers · ~1.53B free tokens/mo · 19 routing strategies · $0 to start."/>
|
||||
<img src="./docs/diagrams/readme-hero.svg" width="100%" alt="OmniRoute — Never stop coding. Every AI tool → 338 providers — 90+ free — through one endpoint. Claude Code, Codex, Cursor, Cline, Copilot & Antigravity into FREE Claude / GPT / Gemini with auto-fallback. RTK + Caveman stacked compression saves 15–95% tokens (~89% avg) — never hit limits. 338 AI providers · 90+ free tiers · ~1.51B free tokens/mo · 19 routing strategies · $0 to start."/>
|
||||
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
|
||||
## 💰 ~1.53B Free Tokens / Month
|
||||
## 💰 ~1.51B Free Tokens / Month
|
||||
|
||||
</div>
|
||||
|
||||
> Stacking free tiers by hand is painful — dozens of SDKs, dozens of rate limits, and no idea how much you actually have. OmniRoute aggregates the **documented** free tiers of **43 provider pools / 516 models** into one honest number and shows it live on the dashboard (`/dashboard/free-tiers`).
|
||||
> Stacking free tiers by hand is painful — dozens of SDKs, dozens of rate limits, and no idea how much you actually have. OmniRoute aggregates the **documented** free tiers of **42 provider pools / 495 models** into one honest number and shows it live on the dashboard (`/dashboard/free-tiers`).
|
||||
|
||||
<img src="./docs/diagrams/free-tier-budget.svg" width="100%" alt="OmniRoute free-tier budget card: ~1.53B free tokens per month steady, up to ~2.15B in the first month with signup credits, from the documented free tiers of 43 provider pools / 516 models behind one endpoint. Honest pool-deduped math — each shared pool counted once (counting every rate limit 24/7 would read ~10B; not published), 15 providers ToS-flagged so you decide. Budget bar of the countable free pools with per-model grid (Mistral Large 3 1B, GPT-4o mini 150M, Gemini 2.5 Flash 60M … Claude Sonnet 4.5 25K), one-time first-month signup credits (vertex 300M, agentrouter 200M, predibase 25M, together 25M, glm-cn 20M, doubao 15M, ai21 10M, longcat 10M, deepseek 5M, hyperbolic 5M, nscale 5M), plus permanently-free no-token-cap providers (SiliconFlow, Z.AI GLM-Flash, Kilo, OpenCode Zen, baidu …) and a $10 OpenRouter top-up unlocking +24M/mo — surfaced separately so they never inflate the headline. Live used/remaining on /dashboard/free-tiers."/>
|
||||
<img src="./docs/diagrams/free-tier-budget.svg" width="100%" alt="OmniRoute free-tier budget card: ~1.51B free tokens per month steady, up to ~2.13B in the first month with signup credits, from the documented free tiers of 42 provider pools / 495 models behind one endpoint. Honest pool-deduped math — each shared pool counted once (counting every rate limit 24/7 would read ~10B; not published), 15 providers ToS-flagged so you decide. Budget bar of the countable free pools with per-model grid (Mistral Large 3 1B, GPT-4o mini 150M, Gemini 2.5 Flash 60M … Claude Sonnet 4.5 25K), one-time first-month signup credits (vertex 300M, agentrouter 200M, predibase 25M, together 25M, glm-cn 20M, doubao 15M, ai21 10M, longcat 10M, deepseek 5M, hyperbolic 5M, nscale 5M), plus permanently-free no-token-cap providers (SiliconFlow, Z.AI GLM-Flash, Kilo, OpenCode Zen, baidu …) and a $10 OpenRouter top-up unlocking +24M/mo — surfaced separately so they never inflate the headline. Live used/remaining on /dashboard/free-tiers."/>
|
||||
|
||||
> Animated summary of the live `/dashboard/free-tiers` page. Full methodology (pool dedupe, credit tiers, provider terms): **[docs/reference/FREE_TIERS.md](docs/reference/FREE_TIERS.md)**.
|
||||
>
|
||||
@@ -81,7 +81,7 @@
|
||||
<tr>
|
||||
<td align="right"><b>⚙️ Features</b></td>
|
||||
<td align="center"><a href="#-combos--the-flagship">🎯 Combos</a></td>
|
||||
<td align="center"><a href="#-291-ai-providers--90-free">🌐 Providers</a></td>
|
||||
<td align="center"><a href="#-338-ai-providers--90-free">🌐 Providers</a></td>
|
||||
<td align="center"><a href="#-full-cli--a2a--mcp">🔌 CLI & MCP</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
@@ -190,7 +190,7 @@ curl http://localhost:20128/v1/chat/completions \
|
||||
|
||||
</div>
|
||||
|
||||
<img src="./docs/diagrams/promise-pillars.svg" width="100%" alt="The Promise — One endpoint. 291 providers. Never stop building — OmniRoute picks the cheapest one that works. Six pillars: Never hit limits (auto-fallback across 291 providers in milliseconds, zero downtime) · Save up to 95% tokens (RTK + Caveman stacked compression cuts 15–95%, ~89% avg on tool-heavy sessions) · $0 to start (90+ free tiers, 40+ free forever — no card needed) · Every tool works (33 coding agents through one config) · One endpoint (OpenAI ↔ Claude ↔ Gemini ↔ Responses API at /v1) · Production-grade (circuit breakers, TLS stealth, MCP 105 tools, A2A, memory, guardrails, evals — 25,000+ tests)."/>
|
||||
<img src="./docs/diagrams/promise-pillars.svg" width="100%" alt="The Promise — One endpoint. 338 providers. Never stop building — OmniRoute picks the cheapest one that works. Six pillars: Never hit limits (auto-fallback across 338 providers in milliseconds, zero downtime) · Save up to 95% tokens (RTK + Caveman stacked compression cuts 15–95%, ~89% avg on tool-heavy sessions) · $0 to start (90+ free tiers, 40+ free forever — no card needed) · Every tool works (33 coding agents through one config) · One endpoint (OpenAI ↔ Claude ↔ Gemini ↔ Responses API at /v1) · Production-grade (circuit breakers, TLS stealth, MCP 105 tools, A2A, memory, guardrails, evals — 25,000+ tests)."/>
|
||||
|
||||
<br/>
|
||||
<br/>
|
||||
@@ -409,7 +409,7 @@ All **19** strategies — mix & match per combo step:
|
||||
<tr>
|
||||
<td align="center">17</td>
|
||||
<td nowrap><code>auto</code></td>
|
||||
<td>12-factor live scoring across every connection 🤖</td>
|
||||
<td>14-factor live scoring across every connection 🤖</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">18</td>
|
||||
@@ -423,7 +423,7 @@ All **19** strategies — mix & match per combo step:
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<sub>The Auto-Combo engine scores every candidate on **12 factors** (health, quota, cost, latency, success rate, freshness…) — see [`docs/routing/AUTO-COMBO.md`](docs/routing/AUTO-COMBO.md).</sub>
|
||||
<sub>The Auto-Combo engine scores every candidate on **14 factors** (health, quota, cost, latency, success rate, freshness…) — see [`docs/routing/AUTO-COMBO.md`](docs/routing/AUTO-COMBO.md).</sub>
|
||||
|
||||
##
|
||||
|
||||
@@ -441,7 +441,7 @@ All **19** strategies — mix & match per combo step:
|
||||
|
||||
</div>
|
||||
|
||||
<img src="./docs/diagrams/comparison-table.svg" width="100%" alt="What sets OmniRoute apart — comparison table vs 9router, OpenRouter, CLIProxyAPI and LiteLLM across 13 capabilities. OmniRoute: 291 providers, 90+ free providers built-in, 19 routing strategies, 12-engine token compression, built-in MCP server with 105 tools, A2A agent protocol, persistent memory, guardrails, cloud agents, TLS fingerprint stealth, Desktop/Termux/PWA, 43 i18n UI locales, 100% MIT self-hosted. OmniRoute is the only one with the full set; competitors show a mix of checks, partials and crosses. Verified from each project's docs."/>
|
||||
<img src="./docs/diagrams/comparison-table.svg" width="100%" alt="What sets OmniRoute apart — comparison table vs 9router, OpenRouter, CLIProxyAPI and LiteLLM across 13 capabilities. OmniRoute: 338 providers, 90+ free providers built-in, 19 routing strategies, 12-engine token compression, built-in MCP server with 105 tools, A2A agent protocol, persistent memory, guardrails, cloud agents, TLS fingerprint stealth, Desktop/Termux/PWA, 43 i18n UI locales, 100% MIT self-hosted. OmniRoute is the only one with the full set; competitors show a mix of checks, partials and crosses. Verified from each project's docs."/>
|
||||
|
||||
<sub>📊 Full methodology & per-feature detail vs 9router, OpenRouter, CLIProxyAPI & LiteLLM → [`docs/comparison/OMNIROUTE_VS_ALTERNATIVES.md`](docs/comparison/OMNIROUTE_VS_ALTERNATIVES.md)</sub>
|
||||
|
||||
@@ -499,8 +499,11 @@ Pix copia-e-cola:
|
||||
|
||||
</div>
|
||||
|
||||
> Recent highlights from **v3.8.20 → v3.8.49**. Full history in [`CHANGELOG.md`](CHANGELOG.md).
|
||||
> Recent highlights from **v3.8.20 → v3.8.50**. Full history in [`CHANGELOG.md`](CHANGELOG.md).
|
||||
|
||||
- **🎛️ OmniConductor** — inbound A2A delegation to your agent fleet, Conductor skills on the Agent Card, and a dashboard panel with Faro push-to-talk voice chat. → [A2A Server](docs/frameworks/A2A-SERVER.md)
|
||||
- **🛂 Adaptive admission & overload protection** — heavyweight chat requests queue instead of 503ing, with atomic RPM rolling leases per connection. → [Resilience Guide](docs/architecture/RESILIENCE_GUIDE.md)
|
||||
- **🗂️ Canonical `/v1/models` ordering** — one contiguous provider-grouped block per provider (combos pinned first), stable across every catalog source. → [API Reference](docs/reference/API_REFERENCE.md)
|
||||
- **🗜️ Compression hardening** — default-on inflation guard, Caveman packs for DE / FR / JA + Chinese (wényán), RTK filters for Gradle & .NET. → [Compression](docs/compression/COMPRESSION_ENGINES.md)
|
||||
- **💸 Honest flat-rate cost** — subscription / coding-plan providers read **$0** in cost analytics; budget, quota & routing keep estimating. → [API Reference](docs/reference/API_REFERENCE.md)
|
||||
- **⚖️ Quota-Share routing** — split a shared account's quota fairly across pooled keys, work-conserving so idle slices are lent out. → [Resilience Guide](docs/architecture/RESILIENCE_GUIDE.md)
|
||||
@@ -515,7 +518,7 @@ Pix copia-e-cola:
|
||||
- **🖼️ New endpoints** — `/v1/ocr` (Mistral OCR) and `/v1/audio/translations` (Whisper-style) round out the media surface. → [API Reference](docs/reference/API_REFERENCE.md)
|
||||
- **🎨 Image / video / audio generation** — one API for media: xAI Grok Imagine & Novita AI video, ComfyUI, Freepik, Adobe Firefly, Microsoft Designer, Google Imagen, Segmind, EdgeTTS. → [API Reference](docs/reference/API_REFERENCE.md)
|
||||
- **🌍 Deployment & ops** — reverse-proxy `basePath`, browser-language auto-detect, per-key device tracking, root-less MITM trust, zh-TW localization. → [Environment](docs/reference/ENVIRONMENT.md)
|
||||
- **🤝 More providers & agents** — Cursor Cloud Agent, Grok Build (xAI) with browser + OAuth login, Ollama first-class card, Claude Opus 5 & Sonnet 5, Kimi official partnership (Code/Web/Moonshot), Zed, Requesty, SenseNova, Yuanbao, Agnes AI… and a refreshed **291-provider catalog**. → [Providers](docs/reference/PROVIDER_REFERENCE.md)
|
||||
- **🤝 More providers & agents** — Cursor Cloud Agent, Grok Build (xAI) with browser + OAuth login, Ollama first-class card, Claude Opus 5 & Sonnet 5, Kimi official partnership (Code/Web/Moonshot), Zed, Requesty, SenseNova, Yuanbao, Agnes AI… and a refreshed **338-provider catalog**. → [Providers](docs/reference/PROVIDER_REFERENCE.md)
|
||||
- **📡 Routing transparency** — every response carries an `X-OmniRoute-Decision` header naming the strategy/provider/latency that served it, a new `cache-optimized` combo strategy + Auto-Combo `cacheAffinity` factor route repeat requests back to the connection holding the cached prefix, and a read-only `/v1/auto-combo/{channel}/candidates` endpoint exposes an `auto/*` channel's live candidate pool. → [Auto-Combo](docs/routing/AUTO-COMBO.md)
|
||||
- **⚡ Local performance & infra** — one-click local Redis, Cloudflare Workers / Deno Deploy relay deployers, Bifrost & Mux as supervised embedded services. → [Embedded Services](docs/frameworks/EMBEDDED-SERVICES.md)
|
||||
|
||||
@@ -576,11 +579,11 @@ Pix copia-e-cola:
|
||||
|
||||
<div align="center">
|
||||
|
||||
## 🌐 291 AI Providers — 90+ Free
|
||||
## 🌐 338 AI Providers — 90+ Free
|
||||
|
||||
</div>
|
||||
|
||||
> The most complete catalog of any open-source router: **291 providers**, **90+ with a free tier**, **40+ free forever**.
|
||||
> The most complete catalog of any open-source router: **338 providers**, **90+ with a free tier**, **40+ free forever**.
|
||||
|
||||
<div align="center">
|
||||
|
||||
@@ -1038,7 +1041,7 @@ same process on one port, so there is no separate CLI-only package today.
|
||||
<tr><td nowrap><b>Runtime</b></td><td>Node.js 22.x / 24.x LTS — <code>>=22.22.2 <23 || >=24.0.0 <27</code></td></tr>
|
||||
<tr><td nowrap><b>Language</b></td><td>TypeScript 6.0 — <b>100% TypeScript</b> across <code>src/</code> and <code>open-sse/</code> (zero <code>any</code> in core since v2.0)</td></tr>
|
||||
<tr><td nowrap><b>Framework</b></td><td>Next.js 16 + React 19 + Tailwind CSS 4</td></tr>
|
||||
<tr><td nowrap><b>Database</b></td><td>better-sqlite3 (SQLite, WAL journaling) + LowDB (JSON legacy) — 95 domain modules, 110 migrations</td></tr>
|
||||
<tr><td nowrap><b>Database</b></td><td>better-sqlite3 (SQLite, WAL journaling) + LowDB (JSON legacy) — 95 domain modules, 144 migrations</td></tr>
|
||||
<tr><td nowrap><b>Memory</b></td><td>SQLite FTS5 full-text + int8-quantized vector embeddings, typed decay</td></tr>
|
||||
<tr><td nowrap><b>Schemas</b></td><td>Zod 4 — MCP tool I/O validation + API contracts</td></tr>
|
||||
<tr><td nowrap><b>Protocols</b></td><td>MCP (stdio / HTTP / SSE) + A2A v0.3 (JSON-RPC 2.0 + SSE)</td></tr>
|
||||
@@ -1099,9 +1102,9 @@ same process on one port, so there is no separate CLI-only package today.
|
||||
<tr><td nowrap><b><a href="docs/compression/COMPRESSION_RULES_FORMAT.md">Compression Rules Format</a></b></td><td>JSON rule-pack schemas for Caveman and RTK filters</td></tr>
|
||||
<tr><td nowrap><b><a href="docs/compression/COMPRESSION_LANGUAGE_PACKS.md">Compression Language Packs</a></b></td><td>Language detection and Caveman rule-pack authoring</td></tr>
|
||||
<tr><td nowrap><b><a href="docs/architecture/RESILIENCE_GUIDE.md">Resilience Guide</a></b></td><td>Circuit breakers, cooldowns, queue, anti-thundering herd, TLS spoofing</td></tr>
|
||||
<tr><td nowrap><b><a href="docs/routing/AUTO-COMBO.md">Auto-Combo Engine</a></b></td><td>12-factor scoring, mode packs, self-healing</td></tr>
|
||||
<tr><td nowrap><b><a href="docs/routing/AUTO-COMBO.md">Auto-Combo Engine</a></b></td><td>14-factor scoring, mode packs, self-healing</td></tr>
|
||||
<tr><td nowrap><b><a href="docs/ops/PROXY_GUIDE.md">Proxy Guide</a></b></td><td>3-level proxy system, 1proxy marketplace, registry CRUD</td></tr>
|
||||
<tr><td nowrap><b><a href="docs/reference/FREE_TIERS.md">Free Tiers</a></b></td><td>25+ free API providers consolidated directory</td></tr>
|
||||
<tr><td nowrap><b><a href="docs/reference/FREE_TIERS.md">Free Tiers</a></b></td><td>90+ free providers consolidated directory (42 documented token pools / 495 models)</td></tr>
|
||||
<tr><td nowrap><b><a href="docs/guides/FEATURES.md">Features Gallery</a></b></td><td>Visual dashboard tour with screenshots</td></tr>
|
||||
<tr><td nowrap><b><a href="docs/architecture/CODEBASE_DOCUMENTATION.md">Codebase Documentation</a></b></td><td>Beginner-friendly codebase walkthrough</td></tr>
|
||||
</table>
|
||||
@@ -1112,7 +1115,7 @@ same process on one port, so there is no separate CLI-only package today.
|
||||
<tr><th align="left">Document</th><th align="left">Description</th></tr>
|
||||
<tr><td nowrap><b><a href="docs/reference/API_REFERENCE.md">API Reference</a></b></td><td>All endpoints with examples</td></tr>
|
||||
<tr><td nowrap><b><a href="docs/openapi.yaml">OpenAPI Spec</a></b></td><td>OpenAPI 3.0 specification</td></tr>
|
||||
<tr><td nowrap><b><a href="open-sse/mcp-server/README.md">MCP Server</a></b></td><td>104 MCP tools, IDE configs, Python/TS/Go clients</td></tr>
|
||||
<tr><td nowrap><b><a href="open-sse/mcp-server/README.md">MCP Server</a></b></td><td>105 MCP tools, IDE configs, Python/TS/Go clients</td></tr>
|
||||
<tr><td nowrap><b><a href="docs/frameworks/MCP-SERVER.md">MCP Server Guide</a></b></td><td>MCP installation, transports, and tool reference</td></tr>
|
||||
<tr><td nowrap><b><a href="src/lib/a2a/README.md">A2A Server</a></b></td><td>JSON-RPC 2.0 protocol, skills, streaming, task mgmt</td></tr>
|
||||
<tr><td nowrap><b><a href="docs/frameworks/A2A-SERVER.md">A2A Server Guide</a></b></td><td>A2A agent card, tasks, skills, and streaming</td></tr>
|
||||
@@ -1126,7 +1129,7 @@ same process on one port, so there is no separate CLI-only package today.
|
||||
<tr><td nowrap><b><a href="docs/ops/BRANCHING_MODEL.md">Branching & Release Model</a></b></td><td>Where PRs target (<code>release/*</code>), what <code>main</code> and tags mean</td></tr>
|
||||
<tr><td nowrap><b><a href="CHANGELOG.md">Changelog</a></b></td><td>Full per-version release history</td></tr>
|
||||
<tr><td nowrap><b><a href="SECURITY.md">Security Policy</a></b></td><td>Vulnerability reporting and security practices</td></tr>
|
||||
<tr><td nowrap><b><a href="docs/guides/I18N.md">i18n Guide</a></b></td><td>40+ language support, translation workflow, RTL</td></tr>
|
||||
<tr><td nowrap><b><a href="docs/guides/I18N.md">i18n Guide</a></b></td><td>43-language support, translation workflow, RTL</td></tr>
|
||||
<tr><td nowrap><b><a href="docs/ops/RELEASE_CHECKLIST.md">Release Checklist</a></b></td><td>Pre-release validation steps</td></tr>
|
||||
<tr><td nowrap><b><a href="docs/ops/COVERAGE_PLAN.md">Coverage Plan</a></b></td><td>Test coverage strategy and 25,000+ test suite</td></tr>
|
||||
</table>
|
||||
@@ -1269,7 +1272,7 @@ A heartfelt thank-you to the people who fund OmniRoute out of their own pocket
|
||||
|
||||
<div align="center">
|
||||
|
||||
## 👥 500+ Contributors
|
||||
## 👥 320+ Contributors
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
@@ -111,7 +111,7 @@ Pluggable subsystems exposed to clients, agents, and operators.
|
||||
|
||||
Combo routing, scoring, and replay.
|
||||
|
||||
- [AUTO-COMBO.md](routing/AUTO-COMBO.md) — Auto-Combo (multi-factor scoring, 17 strategies).
|
||||
- [AUTO-COMBO.md](routing/AUTO-COMBO.md) — Auto-Combo (multi-factor scoring, 19 strategies).
|
||||
- [QUOTA_SHARE.md](routing/QUOTA_SHARE.md) — quota sharing engine.
|
||||
- [REASONING_REPLAY.md](routing/REASONING_REPLAY.md) — reasoning replay cache.
|
||||
|
||||
|
||||
@@ -17,7 +17,7 @@ It provides a single OpenAI-compatible endpoint (`/v1/*`) and routes traffic acr
|
||||
|
||||
Core capabilities:
|
||||
|
||||
- OpenAI-compatible API surface for CLI/tools (271 providers, 89 executors)
|
||||
- OpenAI-compatible API surface for CLI/tools (338 providers, 101 executors)
|
||||
- Request/response translation across provider formats
|
||||
- Model combo fallback (multi-model sequence)
|
||||
- Structured combo steps (`provider + model + connection`) with runtime ordering by `compositeTiers`
|
||||
@@ -365,7 +365,7 @@ relying on a static combo definition. It powers the `auto/*` model prefix family
|
||||
|
||||
Key capabilities:
|
||||
|
||||
- **17 routing strategies** (priority, weighted, fill-first, round-robin, P2C, random,
|
||||
- **19 routing strategies** (priority, weighted, fill-first, round-robin, P2C, random,
|
||||
least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random,
|
||||
**auto**, lkgp, context-optimized, context-relay, **fusion**, plus a fallback path) —
|
||||
auto is the headline addition in v3.8.0; `fusion` (panel fan-out + judge synthesis,
|
||||
|
||||
@@ -452,12 +452,12 @@ open-sse/
|
||||
├── types.d.ts
|
||||
├── config/ Provider registries, header profiles, identity, …
|
||||
├── handlers/ Request handlers (chat, embeddings, audio, image, …)
|
||||
├── executors/ 89 provider-specific HTTP executors
|
||||
├── executors/ 101 provider-specific HTTP executors
|
||||
├── translator/ Format conversion (OpenAI ↔ Claude ↔ Gemini ↔ Cursor ↔ Kiro)
|
||||
├── transformer/ Responses API ↔ Chat Completions stream transformer
|
||||
├── services/ 80+ service modules (combos, fallback, quotas, identity, …)
|
||||
├── utils/ Streaming helpers, TLS client, AWS SigV4, proxy fetch, …
|
||||
└── mcp-server/ MCP server (3 transports, 32 scopes, 99 tools)
|
||||
└── mcp-server/ MCP server (3 transports, 31 scopes, 105 tools)
|
||||
```
|
||||
|
||||
### 4.1 `open-sse/handlers/`
|
||||
@@ -482,7 +482,7 @@ open-sse/
|
||||
|
||||
### 4.2 `open-sse/executors/`
|
||||
|
||||
84 provider executors, each extending `BaseExecutor` (`base.ts`):
|
||||
101 provider executors, each extending `BaseExecutor` (`base.ts`):
|
||||
|
||||
`antigravity`, `azure-openai`, `blackbox-web`, `chatgpt-web`, `cliproxyapi`,
|
||||
`cloudflare-ai`, `codex`, `commandCode`, `cursor`, `default`, `devin-cli`,
|
||||
@@ -491,7 +491,7 @@ open-sse/
|
||||
(shared identity helper) and `index.ts` (registry).
|
||||
|
||||
> Note: providers not listed here are served by `default.ts` using the generic
|
||||
> OpenAI-compatible executor. The full provider catalog (268 entries) lives in
|
||||
> OpenAI-compatible executor. The full provider catalog (338 providers) lives in
|
||||
> `src/shared/constants/providers.ts`.
|
||||
|
||||
### 4.3 `open-sse/translator/`
|
||||
@@ -524,7 +524,7 @@ Highlights (full list under `open-sse/services/`):
|
||||
|
||||
| Concern | Files |
|
||||
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Combo routing | `combo.ts` (17 strategies), `comboConfig.ts`, `comboMetrics.ts`, `comboManifestMetrics.ts`, `comboAgentMiddleware.ts` |
|
||||
| Combo routing | `combo.ts` (19 strategies), `comboConfig.ts`, `comboMetrics.ts`, `comboManifestMetrics.ts`, `comboAgentMiddleware.ts` |
|
||||
| Auto Combo engine | `autoCombo/` — `engine.ts`, `scoring.ts`, `taskFitness.ts`, `virtualFactory.ts`, `modePacks.ts`, `autoPrefix.ts`, `persistence.ts`, `providerDiversity.ts`, `providerRegistryAccessor.ts`, `routerStrategy.ts`, `selfHealing.ts`, `index.ts` |
|
||||
| Resilience | `accountFallback.ts` (cooldown + lockout), `errorClassifier.ts`, `emergencyFallback.ts`, `rateLimitManager.ts`, `rateLimitSemaphore.ts`, `accountSemaphore.ts`, `accountSelector.ts` |
|
||||
| Quotas | `quotaMonitor.ts`, `quotaPreflight.ts`, `bailianQuotaFetcher.ts`, `codexQuotaFetcher.ts`, `deepseekQuotaFetcher.ts`, `openrouterQuotaFetcher.ts`, `openrouterFreeWindow.ts`, `crofUsageFetcher.ts`, `antigravityCredits.ts` |
|
||||
@@ -544,7 +544,7 @@ Highlights (full list under `open-sse/services/`):
|
||||
5 compression tools, 3 memory tools, 4 skills tools, plus advanced tools added
|
||||
through `advancedTools.ts`).
|
||||
- **3 transports**: stdio, HTTP Streamable, SSE.
|
||||
- **13 scopes** declared in `src/shared/constants/mcpScopes.ts`.
|
||||
- **31 scopes** declared in `src/shared/constants/mcpScopes.ts`.
|
||||
- Audit table: `mcp_tool_audit` (populated by `audit.ts`).
|
||||
- Files: `server.ts`, `index.ts`, `httpTransport.ts`, `audit.ts`, `scopeEnforcement.ts`,
|
||||
`runtimeHeartbeat.ts`, `descriptionCompressor.ts`, `schemas/{tools, a2a, audit, index}.ts`,
|
||||
|
||||
@@ -142,5 +142,5 @@ Namespaces added by Group B:
|
||||
| `sidebar.costsOverview` | Costs overview item |
|
||||
| `activity.*` | All Activity page strings (title, verbs, filters, empty state) |
|
||||
|
||||
Source-of-truth locales: `pt-BR` and `en`. All other 39 locales fall back to
|
||||
Source-of-truth locales: `pt-BR` and `en`. All other 41 locales fall back to
|
||||
English via the `next-intl` fallback mechanism (configured in `src/i18n/config.ts`).
|
||||
|
||||
@@ -205,7 +205,7 @@ Runs after `build`. Blocks merge on failure.
|
||||
|
||||
| Suite | Validates | Blocking |
|
||||
| ---------------- | ------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
|
||||
| `test:vitest` | MCP server (94 tools), autoCombo, cache — vitest runner | Yes |
|
||||
| `test:vitest` | MCP server (105 tools), autoCombo, cache — vitest runner | Yes |
|
||||
| `test:vitest:ui` | UI component tests — vitest runner | **Blocking** — pre-existing failures are explicitly excluded in `vitest.config.ts`; new failures fail the job |
|
||||
|
||||
### Nightly workflows (scheduled, advisory)
|
||||
|
||||
@@ -182,7 +182,7 @@ src/
|
||||
| `compliance/` | Audit log + provider audit — see `docs/security/COMPLIANCE.md` |
|
||||
| `compression/` | Compression engine glue (engines live in `open-sse/services/compression/`) |
|
||||
| `config/` | Runtime config helpers |
|
||||
| `db/` | 95+ domain DB modules + 110+ migrations (always go through here for SQLite) |
|
||||
| `db/` | 95+ domain DB modules + 144 migrations (always go through here for SQLite) |
|
||||
| `quota/` | Quota Sharing Engine: `dimensions.ts` (types/Zod), `types.ts` (QuotaStore interface), `sqliteQuotaStore.ts`, `redisQuotaStore.ts`, `storeFactory.ts`, `fairShare.ts`, `burnRate.ts`, `planResolver.ts`, `planRegistry.ts`, `saturationSignals.ts`, `enforce.ts`, `spendRecorder.ts` — see `docs/routing/QUOTA_SHARE.md` |
|
||||
| `radar/` | Radar free-model catalog client: `feedSchema.ts`, `pinnedKeys.ts`, `verify.ts`, `sync.ts`, `applyFeed.ts`, `index.ts` (`getRadarCatalog()`) — see `docs/frameworks/RADAR.md` |
|
||||
| `display/` | UI formatting helpers (cost, latency, etc.) |
|
||||
@@ -209,7 +209,7 @@ src/
|
||||
| `cacheLayer.ts`, `idempotencyLayer.ts` | Request caching + idempotency |
|
||||
| (~30 more top-level files) | Specialized helpers (logEnv, modelsDevSync, piiSanitizer, etc.) |
|
||||
|
||||
### `src/db/` — Database (94 modules + 106 migrations)
|
||||
### `src/db/` — Database (117 modules + 144 migrations)
|
||||
|
||||
| Subdir | Purpose |
|
||||
| ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
@@ -244,9 +244,9 @@ src/
|
||||
|
||||
| Module | Purpose |
|
||||
| -------------------------------- | ---------------------------------------------------------------------- |
|
||||
| `constants/providers.ts` | **236 providers** with Zod validation (source of truth) |
|
||||
| `constants/providers.ts` | **338 providers** with Zod validation (source of truth) |
|
||||
| `constants/cliTools.ts` | External CLI tool registry |
|
||||
| `constants/routingStrategies.ts` | **17 routing strategies** with priorities |
|
||||
| `constants/routingStrategies.ts` | **19 routing strategies** with priorities |
|
||||
| `constants/publicApiRoutes.ts` | Routes that require Bearer (vs management) auth |
|
||||
| `constants/upstreamHeaders.ts` | Header denylist for upstream requests |
|
||||
| `validation/schemas.ts` | ~80 Zod schemas (single source of truth for API contracts) |
|
||||
@@ -270,7 +270,7 @@ open-sse/
|
||||
├── translator/ # Format converters (9 request, 9 response, 9 helpers)
|
||||
├── transformer/ # Responses API ↔ Chat Completions (TransformStream)
|
||||
├── services/ # ~80+ service modules (combo, accountFallback, autoCombo, reasoningCache, claude code/chatgpt stealth, modelDeprecation, taskAwareRouter, workflowFSM, etc.)
|
||||
├── mcp-server/ # MCP server (99 tools, 3 transports, 32 scopes)
|
||||
├── mcp-server/ # MCP server (105 tools, 3 transports, 31 scopes)
|
||||
├── config/ # Provider/model registries, header config, model aliases
|
||||
├── utils/ # TLS client, proxy fetch/dispatcher, network helpers
|
||||
├── index.ts # Workspace entry
|
||||
@@ -398,16 +398,16 @@ open-sse/
|
||||
| `COVERAGE_PLAN.md` | Coverage goals and current state |
|
||||
| `FREE_TIERS.md` | Curated free-tier providers (48+ free + 11 OAuth) |
|
||||
| `CLI-TOOLS.md` | External CLI integrations + Internal OmniRoute CLI |
|
||||
| `I18N.md` | i18n architecture, adding a language, 30 locales |
|
||||
| `I18N.md` | i18n architecture, adding a language, 43 locales |
|
||||
| `UNINSTALL.md` | Clean uninstall steps |
|
||||
| `PROVIDER_REFERENCE.md` | **Auto-generated** catalog of 236 providers (regen: `npm run gen:provider-reference`) |
|
||||
| `PROVIDER_REFERENCE.md` | **Auto-generated** catalog of 338 providers (regen: `npm run gen:provider-reference`) |
|
||||
|
||||
### Subsystem deep-dives
|
||||
|
||||
| Doc | Purpose |
|
||||
| -------------------------- | ------------------------------------------------------------------- |
|
||||
| `MCP-SERVER.md` | MCP server: 99 tools, 3 transports, 32 scopes, REST endpoints |
|
||||
| `A2A-SERVER.md` | A2A v0.3: JSON-RPC, 5 skills, REST helpers, agent card |
|
||||
| `MCP-SERVER.md` | MCP server: 105 tools, 3 transports, 31 scopes, REST endpoints |
|
||||
| `A2A-SERVER.md` | A2A v0.3: JSON-RPC, 6 skills, REST helpers, agent card |
|
||||
| `AGENT_PROTOCOLS_GUIDE.md` | Unified guide: A2A vs ACP vs Cloud Agents |
|
||||
| `CLOUD_AGENT.md` | Codex Cloud / Devin / Jules orchestration |
|
||||
| `SKILLS.md` | Skills framework (built-in + marketplace + SkillsSH + sandbox) |
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
<svg viewBox="0 0 1200 350" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="Animated terminal demoing the OmniRoute CLI: omniroute providers list (278 providers registered, anthropic, codex, glm, kimi shown active), omniroute combo list (always-on priority, cost-saver, fusion-panel, context-relay) and omniroute health (healthy, 18412 requests in 24h, p95 412ms, circuit breakers 24 closed, 1 half-open, 0 open), cycling over the 80+ command surface: providers, oauth, keys, combo, nodes, models, cache, compression, cost, usage, quota, health, resilience, telemetry, logs, audit, mcp, a2a, cloud, memory, skills, eval, doctor, repl, tunnel, backup, sync, webhooks, policy, pricing, translator, simulate and more.">
|
||||
<svg viewBox="0 0 1200 350" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="Animated terminal demoing the OmniRoute CLI: omniroute providers list (338 providers registered, anthropic, codex, glm, kimi shown active), omniroute combo list (always-on priority, cost-saver, fusion-panel, context-relay) and omniroute health (healthy, 18412 requests in 24h, p95 412ms, circuit breakers 24 closed, 1 half-open, 0 open), cycling over the 80+ command surface: providers, oauth, keys, combo, nodes, models, cache, compression, cost, usage, quota, health, resilience, telemetry, logs, audit, mcp, a2a, cloud, memory, skills, eval, doctor, repl, tunnel, backup, sync, webhooks, policy, pricing, translator, simulate and more.">
|
||||
<desc>Compact animated terminal cycling three real OmniRoute CLI commands with a typewriter effect and a scrolling subcommand ticker; the first frame shows the completed providers-list screen.</desc>
|
||||
<defs><clipPath id="tickerClip"><rect x="12" y="304" width="1176" height="40"/></clipPath><clipPath id="tw0"><rect x="64" y="46" height="26" width="0"><animate attributeName="width" calcMode="discrete" values="0;31;61;92;122;153;184;214;245;245" keyTimes="0;0.012;0.018;0.024;0.030;0.036;0.042;0.048;0.054;1" dur="18s" repeatCount="indefinite"/></rect></clipPath><clipPath id="tw1"><rect x="64" y="46" height="26" width="0"><animate attributeName="width" calcMode="discrete" values="0;26;51;76;102;128;153;178;204;204" keyTimes="0;0.345;0.351;0.357;0.363;0.369;0.375;0.381;0.387;1" dur="18s" repeatCount="indefinite"/></rect></clipPath><clipPath id="tw2"><rect x="64" y="46" height="26" width="0"><animate attributeName="width" calcMode="discrete" values="0;20;41;61;82;102;122;143;163;163" keyTimes="0;0.678;0.684;0.690;0.696;0.702;0.708;0.714;0.720;1" dur="18s" repeatCount="indefinite"/></rect></clipPath></defs>
|
||||
<rect width="1200" height="350" fill="#0d1117"/>
|
||||
@@ -6,7 +6,7 @@
|
||||
<path d="M 0 34 L 1200 34" stroke="#ffffff" stroke-opacity="0.08" stroke-width="1"/>
|
||||
<circle cx="24" cy="17" r="6" fill="#ff5f56"/><circle cx="46" cy="17" r="6" fill="#ffbd2e"/><circle cx="68" cy="17" r="6" fill="#27c93f"/>
|
||||
<text x="600" y="22" text-anchor="middle" font-family="Consolas, 'Courier New', monospace" font-size="13" fill="#71717a">omniroute — 80+ commands</text>
|
||||
<g font-family="Consolas, 'Courier New', monospace" font-size="17"><animate attributeName="opacity" values="1;0;0" keyTimes="0;0.006;1" dur="18s" repeatCount="indefinite"/><text x="64" y="66" fill="#F7F6FC">omniroute providers list</text><text x="40" y="100" font-weight="700" fill="#38bdf8">OmniRoute Providers</text><text x="40" y="128" fill="#a1a1aa">1f3a9c2e  anthropic   Claude Max 20x    <tspan fill='#22c55e'>active</tspan></text><text x="40" y="154" fill="#a1a1aa">8c2d5b1a  codex       Codex Pro (team)  <tspan fill='#22c55e'>active</tspan></text><text x="40" y="180" fill="#a1a1aa">f4e0a97b  glm         GLM Coding Plan   <tspan fill='#22c55e'>active</tspan></text><text x="40" y="206" fill="#a1a1aa">03bd6e5f  kimi        Kimi K2 free      <tspan fill='#22c55e'>active</tspan></text><text x="40" y="232" fill="#71717a">… 264 more providers</text></g><g opacity="1" font-family="Consolas, 'Courier New', monospace" font-size="17">
|
||||
<g font-family="Consolas, 'Courier New', monospace" font-size="17"><animate attributeName="opacity" values="1;0;0" keyTimes="0;0.006;1" dur="18s" repeatCount="indefinite"/><text x="64" y="66" fill="#F7F6FC">omniroute providers list</text><text x="40" y="100" font-weight="700" fill="#38bdf8">OmniRoute Providers</text><text x="40" y="128" fill="#a1a1aa">1f3a9c2e  anthropic   Claude Max 20x    <tspan fill='#22c55e'>active</tspan></text><text x="40" y="154" fill="#a1a1aa">8c2d5b1a  codex       Codex Pro (team)  <tspan fill='#22c55e'>active</tspan></text><text x="40" y="180" fill="#a1a1aa">f4e0a97b  glm         GLM Coding Plan   <tspan fill='#22c55e'>active</tspan></text><text x="40" y="206" fill="#a1a1aa">03bd6e5f  kimi        Kimi K2 free      <tspan fill='#22c55e'>active</tspan></text><text x="40" y="232" fill="#71717a">… 334 more providers</text></g><g opacity="1" font-family="Consolas, 'Courier New', monospace" font-size="17">
|
||||
<animate attributeName="opacity" values="1;1;0;0" keyTimes="0;0.315;0.33;1" dur="18s" repeatCount="indefinite"/>
|
||||
<text x="40" y="66" fill="#22c55e">$</text>
|
||||
<g clip-path="url(#tw0)"><text x="64" y="66" fill="#F7F6FC">omniroute providers list</text></g>
|
||||
@@ -14,7 +14,7 @@
|
||||
<animate attributeName="x" calcMode="discrete" values="64;95;125;156;186;217;248;278;309;309" keyTimes="0.000;0.012;0.018;0.024;0.030;0.036;0.042;0.048;0.054;1" dur="18s" repeatCount="indefinite"/>
|
||||
<animate attributeName="opacity" values="0;0;1;0.2;1;0.2;1;0;0" keyTimes="0;0.011;0.012;0.022;0.032;0.042;0.052;0.074;1" dur="18s" repeatCount="indefinite"/>
|
||||
</rect>
|
||||
<text x="40" y="100" font-weight="700" fill="#38bdf8" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.045;0.047" dur="18s" repeatCount="indefinite"/>OmniRoute Providers</text><text x="40" y="128" fill="#a1a1aa" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.053;0.055" dur="18s" repeatCount="indefinite"/>1f3a9c2e  anthropic   Claude Max 20x    <tspan fill='#22c55e'>active</tspan></text><text x="40" y="154" fill="#a1a1aa" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.061;0.063" dur="18s" repeatCount="indefinite"/>8c2d5b1a  codex       Codex Pro (team)  <tspan fill='#22c55e'>active</tspan></text><text x="40" y="180" fill="#a1a1aa" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.069;0.07100000000000001" dur="18s" repeatCount="indefinite"/>f4e0a97b  glm         GLM Coding Plan   <tspan fill='#22c55e'>active</tspan></text><text x="40" y="206" fill="#a1a1aa" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.077;0.079" dur="18s" repeatCount="indefinite"/>03bd6e5f  kimi        Kimi K2 free      <tspan fill='#22c55e'>active</tspan></text><text x="40" y="232" fill="#71717a" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.085;0.08700000000000001" dur="18s" repeatCount="indefinite"/>… 264 more providers</text>
|
||||
<text x="40" y="100" font-weight="700" fill="#38bdf8" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.045;0.047" dur="18s" repeatCount="indefinite"/>OmniRoute Providers</text><text x="40" y="128" fill="#a1a1aa" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.053;0.055" dur="18s" repeatCount="indefinite"/>1f3a9c2e  anthropic   Claude Max 20x    <tspan fill='#22c55e'>active</tspan></text><text x="40" y="154" fill="#a1a1aa" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.061;0.063" dur="18s" repeatCount="indefinite"/>8c2d5b1a  codex       Codex Pro (team)  <tspan fill='#22c55e'>active</tspan></text><text x="40" y="180" fill="#a1a1aa" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.069;0.07100000000000001" dur="18s" repeatCount="indefinite"/>f4e0a97b  glm         GLM Coding Plan   <tspan fill='#22c55e'>active</tspan></text><text x="40" y="206" fill="#a1a1aa" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.077;0.079" dur="18s" repeatCount="indefinite"/>03bd6e5f  kimi        Kimi K2 free      <tspan fill='#22c55e'>active</tspan></text><text x="40" y="232" fill="#71717a" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.085;0.08700000000000001" dur="18s" repeatCount="indefinite"/>… 334 more providers</text>
|
||||
</g><g opacity="0" font-family="Consolas, 'Courier New', monospace" font-size="17">
|
||||
<animate attributeName="opacity" values="0;0;1;1;0;0" keyTimes="0;0.333;0.34500000000000003;0.648;0.663;1" dur="18s" repeatCount="indefinite"/>
|
||||
<text x="40" y="66" fill="#22c55e">$</text>
|
||||
@@ -32,7 +32,7 @@
|
||||
<animate attributeName="x" calcMode="discrete" values="64;84;105;125;146;166;186;207;227;227" keyTimes="0;0.678;0.684;0.690;0.696;0.702;0.708;0.714;0.720;1" dur="18s" repeatCount="indefinite"/>
|
||||
<animate attributeName="opacity" values="0;0;1;0.2;1;0.2;1;0;0" keyTimes="0;0.677;0.678;0.688;0.698;0.708;0.718;0.74;1" dur="18s" repeatCount="indefinite"/>
|
||||
</rect>
|
||||
<text x="40" y="100" font-weight="700" fill="#38bdf8" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.711;0.713" dur="18s" repeatCount="indefinite"/>OmniRoute Health</text><text x="40" y="128" fill="#a1a1aa" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.719;0.721" dur="18s" repeatCount="indefinite"/>  Status: <tspan fill='#22c55e'>healthy</tspan>   Uptime: 4d 12h 33m</text><text x="40" y="154" fill="#a1a1aa" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.727;0.729" dur="18s" repeatCount="indefinite"/>  Requests (24h): 18,412   p95: 412ms</text><text x="40" y="180" fill="#a1a1aa" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.735;0.737" dur="18s" repeatCount="indefinite"/>  Breakers: <tspan fill='#22c55e'>● 24 closed</tspan>  <tspan fill='#f59e0b'>◒ 1 half-open</tspan>  <tspan fill='#ef4444'>○ 0 open</tspan></text><text x="40" y="206" fill="#a1a1aa" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.743;0.745" dur="18s" repeatCount="indefinite"/>  Providers: 278 registered   90+ free tiers</text><text x="40" y="232" fill="#71717a" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.751;0.753" dur="18s" repeatCount="indefinite"/>… live: /dashboard · omniroute status</text>
|
||||
<text x="40" y="100" font-weight="700" fill="#38bdf8" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.711;0.713" dur="18s" repeatCount="indefinite"/>OmniRoute Health</text><text x="40" y="128" fill="#a1a1aa" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.719;0.721" dur="18s" repeatCount="indefinite"/>  Status: <tspan fill='#22c55e'>healthy</tspan>   Uptime: 4d 12h 33m</text><text x="40" y="154" fill="#a1a1aa" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.727;0.729" dur="18s" repeatCount="indefinite"/>  Requests (24h): 18,412   p95: 412ms</text><text x="40" y="180" fill="#a1a1aa" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.735;0.737" dur="18s" repeatCount="indefinite"/>  Breakers: <tspan fill='#22c55e'>● 24 closed</tspan>  <tspan fill='#f59e0b'>◒ 1 half-open</tspan>  <tspan fill='#ef4444'>○ 0 open</tspan></text><text x="40" y="206" fill="#a1a1aa" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.743;0.745" dur="18s" repeatCount="indefinite"/>  Providers: 338 registered   90+ free tiers</text><text x="40" y="232" fill="#71717a" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.751;0.753" dur="18s" repeatCount="indefinite"/>… live: /dashboard · omniroute status</text>
|
||||
</g>
|
||||
<path d="M 0 300 L 1200 300" stroke="#ffffff" stroke-opacity="0.08" stroke-width="1"/>
|
||||
<g clip-path="url(#tickerClip)"><g font-family="Consolas, 'Courier New', monospace" font-size="14" fill="#71717a">
|
||||
|
||||
|
Before Width: | Height: | Size: 12 KiB After Width: | Height: | Size: 12 KiB |
@@ -1,4 +1,4 @@
|
||||
<svg viewBox="0 0 1200 780" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="Comparison table: OmniRoute versus 9router, OpenRouter, CLIProxyAPI and LiteLLM across 13 capabilities. OmniRoute is the only one with the full set: 290 providers, 90+ free providers built-in, 19 routing strategies, 12-engine token compression, a built-in MCP server with 104 tools, A2A protocol, persistent memory, guardrails, cloud agents, TLS fingerprint stealth, desktop/Termux/PWA, 43 UI locales and 100% MIT self-hosted. 9router has free providers, RTK compression and translation but no MCP, A2A, memory, guardrails, cloud agents or stealth. OpenRouter is a hosted SaaS with 400+ models, guardrails and a hosted MCP but is not self-hosted and lacks A2A, memory, cloud agents and stealth. CLIProxyAPI is a light OAuth proxy with two routing strategies. LiteLLM has 100+ providers, A2A and extensive guardrails but no memory, compression, free tier, stealth or cloud agents.">
|
||||
<svg viewBox="0 0 1200 780" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="Comparison table: OmniRoute versus 9router, OpenRouter, CLIProxyAPI and LiteLLM across 13 capabilities. OmniRoute is the only one with the full set: 338 providers, 90+ free providers built-in, 19 routing strategies, 12-engine token compression, a built-in MCP server with 105 tools, A2A protocol, persistent memory, guardrails, cloud agents, TLS fingerprint stealth, desktop/Termux/PWA, 43 UI locales and 100% MIT self-hosted. 9router has free providers, RTK compression and translation but no MCP, A2A, memory, guardrails, cloud agents or stealth. OpenRouter is a hosted SaaS with 400+ models, guardrails and a hosted MCP but is not self-hosted and lacks A2A, memory, cloud agents and stealth. CLIProxyAPI is a light OAuth proxy with two routing strategies. LiteLLM has 100+ providers, A2A and extensive guardrails but no memory, compression, free tier, stealth or cloud agents.">
|
||||
<desc>Static-header comparison table where each capability row fades in top to bottom; the OmniRoute column is highlighted and shows a check or a leading value in every row, while competitors show a mix of checks, partials and crosses.</desc>
|
||||
<defs>
|
||||
<pattern id="gC" width="32" height="32" patternUnits="userSpaceOnUse"><path d="M 32 0 L 0 0 0 32" fill="none" stroke="#ffffff" stroke-opacity="0.05" stroke-width="1"/></pattern>
|
||||
@@ -23,7 +23,7 @@
|
||||
<g font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif">
|
||||
<g opacity="0"><animate attributeName="opacity" values="0;1" dur="0.4s" begin="0.15s" fill="freeze"/>
|
||||
<text x="44" y="196" font-size="14.5" fill="#c9d1d9">Providers</text>
|
||||
<text x="440" y="196" text-anchor="middle" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="15" font-weight="800" fill="#7ee787">290</text>
|
||||
<text x="440" y="196" text-anchor="middle" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="15" font-weight="800" fill="#7ee787">338</text>
|
||||
<text x="604" y="196" text-anchor="middle" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="13.5" font-weight="600" fill="#8b949e">40+</text>
|
||||
<text x="760" y="196" text-anchor="middle" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="13.5" font-weight="600" fill="#8b949e">400+*</text>
|
||||
<text x="916" y="196" text-anchor="middle" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="13.5" font-weight="600" fill="#8b949e">~5</text>
|
||||
@@ -57,7 +57,7 @@
|
||||
</g>
|
||||
<g opacity="0"><animate attributeName="opacity" values="0;1" dur="0.4s" begin="0.51s" fill="freeze"/>
|
||||
<text x="44" y="364" font-size="14.5" fill="#c9d1d9">Built-in MCP server (own tools)</text>
|
||||
<text x="440" y="364" text-anchor="middle" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="15" font-weight="800" fill="#7ee787">104</text>
|
||||
<text x="440" y="364" text-anchor="middle" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="15" font-weight="800" fill="#7ee787">105</text>
|
||||
<use href="#no" x="604" y="359"/>
|
||||
<use href="#mid" x="760" y="359"/>
|
||||
<use href="#no" x="916" y="359"/>
|
||||
|
||||
|
Before Width: | Height: | Size: 13 KiB After Width: | Height: | Size: 13 KiB |
@@ -1,4 +1,4 @@
|
||||
<svg viewBox="0 0 1200 842" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="OmniRoute free-tier budget: about 1.53 billion free tokens per month steady, up to about 2.15 billion in your first month with signup credits, aggregated from the documented free tiers of 43 provider pools and 460+ models behind one endpoint, live on /dashboard/free-tiers. Honest pool-deduped math: each shared free pool is counted once — counting every rate limit 24/7 would read about 10B, which we don't publish; 15 providers carry a ToS flag so you decide. Budget bar of the 19 countable free pools with per-model breakdown: Mistral Large 3 1B, GPT-4o mini 150M, Gemini 2.5 Flash 60M, GLM 4.7 30M, Llama 3.3 70B 30M, Grok-3 24M, DeepSeek V4 Pro 20M, GPT-4.1 18M, Llama 4 Scout 15M, GPT-4o 7M, MiniMax-M2.7 6M, Arcee Trinity 5M, and more. First month adds one-time signup credits of about 626M (vertex 300M, agentrouter 200M, predibase 25M, together 25M, glm-cn 20M, doubao 15M, ai21 10M, longcat 10M, deepseek 5M, hyperbolic 5M, nscale 5M). Plus the un-countable: permanently-free no-token-cap providers (SiliconFlow, Z.AI GLM-Flash, Kilo, OpenCode Zen, baidu and more) and a $10 OpenRouter top-up unlocking +24M per month, surfaced separately so they never inflate the headline. Live used/remaining and per-model breakdown on the dashboard.">
|
||||
<svg viewBox="0 0 1200 842" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="OmniRoute free-tier budget: about 1.51 billion free tokens per month steady, up to about 2.13 billion in your first month with signup credits, aggregated from the documented free tiers of 42 provider pools and 495 models behind one endpoint, live on /dashboard/free-tiers. Honest pool-deduped math: each shared free pool is counted once — counting every rate limit 24/7 would read about 10B, which we don't publish; 15 providers carry a ToS flag so you decide. Budget bar of the 19 countable free pools with per-model breakdown: Mistral Large 3 1B, GPT-4o mini 150M, Gemini 2.5 Flash 60M, GLM 4.7 30M, Llama 3.3 70B 30M, Grok-3 24M, DeepSeek V4 Pro 20M, GPT-4.1 18M, Llama 4 Scout 15M, GPT-4o 7M, MiniMax-M2.7 6M, Arcee Trinity 5M, and more. First month adds one-time signup credits of about 626M (vertex 300M, agentrouter 200M, predibase 25M, together 25M, glm-cn 20M, doubao 15M, ai21 10M, longcat 10M, deepseek 5M, hyperbolic 5M, nscale 5M). Plus the un-countable: permanently-free no-token-cap providers (SiliconFlow, Z.AI GLM-Flash, Kilo, OpenCode Zen, baidu and more) and a $10 OpenRouter top-up unlocking +24M per month, surfaced separately so they never inflate the headline. Live used/remaining and per-model breakdown on the dashboard.">
|
||||
<defs>
|
||||
<pattern id="gridPaperF" width="32" height="32" patternUnits="userSpaceOnUse">
|
||||
<path d="M 32 0 L 0 0 0 32" fill="none" stroke="#ffffff" stroke-opacity="0.06" stroke-width="1"/>
|
||||
@@ -60,10 +60,10 @@
|
||||
<animate attributeName="opacity" values="0;1;1;0" dur="2.4s" begin="1.6s" repeatCount="indefinite"/>
|
||||
</circle>
|
||||
</g>
|
||||
<text x="60" y="228" font-family="Consolas, 'Courier New', monospace" font-size="104" font-weight="800" fill="url(#gradBrandF)">~1.53B</text>
|
||||
<text x="60" y="228" font-family="Consolas, 'Courier New', monospace" font-size="104" font-weight="800" fill="url(#gradBrandF)">~1.51B</text>
|
||||
<text x="62" y="266" font-family="Consolas, 'Courier New', monospace" font-size="15" letter-spacing="3" font-weight="700" fill="#a1a1aa">FREE TOKENS / MONTH · <tspan fill="#22c55e">STEADY</tspan></text>
|
||||
<text x="62" y="298" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="16" fill="#F7F6FC">up to <tspan font-weight="800" fill="#22c55e">~2.15B</tspan> in your first month — signup credits</text>
|
||||
<text x="62" y="326" font-family="Consolas, 'Courier New', monospace" font-size="12" fill="#71717a">documented free tiers · <tspan fill="#8b5cf6">43 provider pools</tspan> · <tspan fill="#8b5cf6">460+ models</tspan> · one endpoint</text>
|
||||
<text x="62" y="298" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="16" fill="#F7F6FC">up to <tspan font-weight="800" fill="#22c55e">~2.13B</tspan> in your first month — signup credits</text>
|
||||
<text x="62" y="326" font-family="Consolas, 'Courier New', monospace" font-size="12" fill="#71717a">documented free tiers · <tspan fill="#8b5cf6">42 provider pools</tspan> · <tspan fill="#8b5cf6">495 models</tspan> · one endpoint</text>
|
||||
|
||||
<!-- ═══ Panel · The honest math ═══ -->
|
||||
<rect x="680" y="84" width="460" height="216" rx="14" fill="#161b22" stroke="#ffffff" stroke-opacity="0.08" stroke-width="1"/>
|
||||
@@ -74,7 +74,7 @@
|
||||
</line>
|
||||
<text x="836" y="156" font-family="Consolas, 'Courier New', monospace" font-size="11.5" fill="#a1a1aa">every rate limit · 24/7</text>
|
||||
<text x="836" y="176" font-family="Consolas, 'Courier New', monospace" font-size="11.5" fill="#ef4444" opacity="0.85">we don't publish that</text>
|
||||
<text x="704" y="240" font-family="Consolas, 'Courier New', monospace" font-size="34" font-weight="800" fill="#22c55e">~1.53B</text>
|
||||
<text x="704" y="240" font-family="Consolas, 'Courier New', monospace" font-size="34" font-weight="800" fill="#22c55e">~1.51B</text>
|
||||
<text x="836" y="224" font-family="Consolas, 'Courier New', monospace" font-size="11.5" fill="#a1a1aa">each shared free pool</text>
|
||||
<text x="836" y="244" font-family="Consolas, 'Courier New', monospace" font-size="11.5" fill="#22c55e">counted once ✓</text>
|
||||
<text x="704" y="280" font-family="Consolas, 'Courier New', monospace" font-size="12" fill="#f59e0b"><tspan font-weight="800">15 providers</tspan> ToS-flagged <tspan fill="#71717a">— we flag it · you decide</tspan></text>
|
||||
|
||||
|
Before Width: | Height: | Size: 18 KiB After Width: | Height: | Size: 18 KiB |
@@ -1,4 +1,4 @@
|
||||
<svg viewBox="0 0 1200 540" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="The OmniRoute promise: one endpoint, 290 providers — never stop building, OmniRoute picks the cheapest one that works. Six pillars. Never hit limits: auto-fallback across 290 providers in milliseconds, quota out means the next provider takes over with zero downtime. Save up to 95 percent of tokens: RTK plus Caveman stacked compression cuts 15 to 95 percent of eligible tokens, about 89 percent average on tool-heavy sessions. Zero dollars to start: 90+ providers with a free tier, 40+ free forever — Qoder, Pollinations, Cloudflare, SiliconFlow — no card needed. Every tool works: 33 coding agents including Claude Code, Codex, Cursor, Cline, Copilot and Antigravity through one config. One endpoint: OpenAI, Claude, Gemini and Responses API translation — point any tool at /v1 and it just works. Production-grade: circuit breakers, TLS stealth, MCP with 104 tools, A2A, memory, guardrails, evals — 25,000+ tests.">
|
||||
<svg viewBox="0 0 1200 540" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="The OmniRoute promise: one endpoint, 338 providers — never stop building, OmniRoute picks the cheapest one that works. Six pillars. Never hit limits: auto-fallback across 338 providers in milliseconds, quota out means the next provider takes over with zero downtime. Save up to 95 percent of tokens: RTK plus Caveman stacked compression cuts 15 to 95 percent of eligible tokens, about 89 percent average on tool-heavy sessions. Zero dollars to start: 90+ providers with a free tier, 40+ free forever — Qoder, Pollinations, Cloudflare, SiliconFlow — no card needed. Every tool works: 33 coding agents including Claude Code, Codex, Cursor, Cline, Copilot and Antigravity through one config. One endpoint: OpenAI, Claude, Gemini and Responses API translation — point any tool at /v1 and it just works. Production-grade: circuit breakers, TLS stealth, MCP with 105 tools, A2A, memory, guardrails, evals — 25,000+ tests.">
|
||||
<desc>Animated promise card: six pillar tiles fade in in reading order, then a soft colored border highlight sweeps from tile to tile in a continuous cycle.</desc>
|
||||
<defs>
|
||||
<pattern id="gridPaperP" width="32" height="32" patternUnits="userSpaceOnUse">
|
||||
@@ -21,7 +21,7 @@
|
||||
<line x1="150" y1="53" x2="1160" y2="53" stroke="#232b38" stroke-width="1.5"/>
|
||||
</g>
|
||||
<g>
|
||||
<text x="40" y="100" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="23" font-weight="600" fill="#c9d1d9">One endpoint. <tspan fill="#a78bfa" font-weight="800">290 providers.</tspan> Never stop building — OmniRoute picks <tspan fill="#7ee787" font-weight="700">the cheapest one that works</tspan>.</text>
|
||||
<text x="40" y="100" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="23" font-weight="600" fill="#c9d1d9">One endpoint. <tspan fill="#a78bfa" font-weight="800">338 providers.</tspan> Never stop building — OmniRoute picks <tspan fill="#7ee787" font-weight="700">the cheapest one that works</tspan>.</text>
|
||||
</g>
|
||||
|
||||
<g font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif">
|
||||
@@ -38,7 +38,7 @@
|
||||
<line x1="3.9" y1="3.9" x2="18.1" y2="18.1"/>
|
||||
</g>
|
||||
<text x="102" y="170" font-size="18" font-weight="800" fill="#74b9ff">Never hit limits</text>
|
||||
<text x="66" y="204" font-size="13.5" fill="#a1a1aa">Auto-fallback across 290 providers in</text>
|
||||
<text x="66" y="204" font-size="13.5" fill="#a1a1aa">Auto-fallback across 338 providers in</text>
|
||||
<text x="66" y="226" font-size="13.5" fill="#a1a1aa">milliseconds. Quota out? The next provider</text>
|
||||
<text x="66" y="248" font-size="13.5" fill="#a1a1aa">takes over — zero downtime.</text>
|
||||
</g>
|
||||
@@ -125,7 +125,7 @@
|
||||
<path d="M 7,11 L 10,14 L 15.4,8.2"/>
|
||||
</g>
|
||||
<text x="862" y="354" font-size="18" font-weight="800" fill="#7ee787">Production-grade</text>
|
||||
<text x="826" y="388" font-size="13.5" fill="#a1a1aa">Circuit breakers, TLS stealth, MCP (104</text>
|
||||
<text x="826" y="388" font-size="13.5" fill="#a1a1aa">Circuit breakers, TLS stealth, MCP (105</text>
|
||||
<text x="826" y="410" font-size="13.5" fill="#a1a1aa">tools), A2A, memory, guardrails, evals —</text>
|
||||
<text x="826" y="432" font-size="13.5" fill="#a1a1aa">25,000+ tests.</text>
|
||||
</g>
|
||||
|
||||
|
Before Width: | Height: | Size: 10 KiB After Width: | Height: | Size: 10 KiB |
@@ -1,4 +1,4 @@
|
||||
<svg viewBox="0 0 1200 548" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="OmniRoute hero: Never stop coding. Every AI tool to 290 providers — 90+ free — through one endpoint. Claude Code, Codex, Cursor, Cline, Copilot and Antigravity into free Claude, GPT and Gemini with auto-fallback. RTK + Caveman stacked compression saves 15 to 95 percent of tokens — about 89 percent average on tool-heavy sessions — so you never hit limits. Stats: 290 AI providers, 90+ free tiers, about 1.53B free tokens per month, 15 to 95 percent token savings, 19 routing strategies, zero dollars to start.">
|
||||
<svg viewBox="0 0 1200 548" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="OmniRoute hero: Never stop coding. Every AI tool to 338 providers — 90+ free — through one endpoint. Claude Code, Codex, Cursor, Cline, Copilot and Antigravity into free Claude, GPT and Gemini with auto-fallback. RTK + Caveman stacked compression saves 15 to 95 percent of tokens — about 89 percent average on tool-heavy sessions — so you never hit limits. Stats: 338 AI providers, 90+ free tiers, about 1.51B free tokens per month, 15 to 95 percent token savings, 19 routing strategies, zero dollars to start.">
|
||||
<desc>Animated hero card: a pulse travels the divider line and a compression bar demo repeatedly shrinks a prompt by up to 95 percent; all headline content is static and readable on the first frame.</desc>
|
||||
<defs>
|
||||
<pattern id="gridPaperH" width="32" height="32" patternUnits="userSpaceOnUse">
|
||||
@@ -28,7 +28,7 @@
|
||||
<text x="48" y="138" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="60" font-weight="800" fill="#e9edf3">Never stop coding<tspan fill="#a855f7">.</tspan></text>
|
||||
|
||||
<!-- subheadline -->
|
||||
<text x="48" y="184" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="25" font-weight="600" fill="#c9d1d9">Every AI tool → <tspan fill="#a78bfa" font-weight="800">290 providers</tspan> — <tspan fill="#7ee787" font-weight="800">90+ free</tspan> — through one endpoint.</text>
|
||||
<text x="48" y="184" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="25" font-weight="600" fill="#c9d1d9">Every AI tool → <tspan fill="#a78bfa" font-weight="800">338 providers</tspan> — <tspan fill="#7ee787" font-weight="800">90+ free</tspan> — through one endpoint.</text>
|
||||
|
||||
<!-- plug line -->
|
||||
<text x="48" y="222" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="16.5" fill="#a1a1aa">Claude Code · Codex · Cursor · Cline · Copilot · Antigravity  →  <tspan fill="#7ee787" font-weight="700">FREE</tspan> Claude / GPT / Gemini · auto-fallback</text>
|
||||
@@ -66,13 +66,13 @@
|
||||
<!-- stat chips -->
|
||||
<g font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" text-anchor="middle">
|
||||
<rect x="48" y="448" width="172" height="52" rx="12" fill="#161b22" stroke="#6c5ce7" stroke-opacity="0.55" stroke-width="1.5"/>
|
||||
<text x="134" y="471" font-size="17" font-weight="800" fill="#a78bfa">290</text>
|
||||
<text x="134" y="471" font-size="17" font-weight="800" fill="#a78bfa">338</text>
|
||||
<text x="134" y="490" font-size="11" fill="#a1a1aa">AI PROVIDERS</text>
|
||||
<rect x="234" y="448" width="172" height="52" rx="12" fill="#161b22" stroke="#22c55e" stroke-opacity="0.55" stroke-width="1.5"/>
|
||||
<text x="320" y="471" font-size="17" font-weight="800" fill="#7ee787">90+</text>
|
||||
<text x="320" y="490" font-size="11" fill="#a1a1aa">FREE TIERS</text>
|
||||
<rect x="420" y="448" width="172" height="52" rx="12" fill="#161b22" stroke="#22c55e" stroke-opacity="0.55" stroke-width="1.5"/>
|
||||
<text x="506" y="471" font-size="17" font-weight="800" fill="#7ee787">~1.53B</text>
|
||||
<text x="506" y="471" font-size="17" font-weight="800" fill="#7ee787">~1.51B</text>
|
||||
<text x="506" y="490" font-size="11" fill="#a1a1aa">FREE TOKENS / MO</text>
|
||||
<rect x="606" y="448" width="172" height="52" rx="12" fill="#161b22" stroke="#e17055" stroke-opacity="0.55" stroke-width="1.5"/>
|
||||
<text x="692" y="471" font-size="17" font-weight="800" fill="#e17055">15–95%</text>
|
||||
|
||||
|
Before Width: | Height: | Size: 7.3 KiB After Width: | Height: | Size: 7.3 KiB |
@@ -89,7 +89,7 @@
|
||||
</circle>
|
||||
</g>
|
||||
<text x="364" y="216" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="20" font-weight="800" fill="url(#gradBrand)">OmniRoute — Smart Router</text>
|
||||
<text x="364" y="243" font-family="Consolas, 'Courier New', monospace" font-size="11.5" fill="#a1a1aa"><tspan fill="#8b5cf6">RTK + Caveman compression</tspan> · <tspan fill="#8b5cf6">18 routing strategies</tspan></text>
|
||||
<text x="364" y="243" font-family="Consolas, 'Courier New', monospace" font-size="11.5" fill="#a1a1aa"><tspan fill="#8b5cf6">RTK + Caveman compression</tspan> · <tspan fill="#8b5cf6">19 routing strategies</tspan></text>
|
||||
<text x="364" y="264" font-family="Consolas, 'Courier New', monospace" font-size="11.5" fill="#a1a1aa">Circuit breakers · TLS stealth · MCP · A2A · Guardrails</text>
|
||||
|
||||
<!-- fan-out topology (always visible) -->
|
||||
|
||||
|
Before Width: | Height: | Size: 17 KiB After Width: | Height: | Size: 17 KiB |
@@ -543,7 +543,7 @@ const agents = detectInstalledAgents();
|
||||
## What's Next?
|
||||
|
||||
- **[API Reference](../reference/API_REFERENCE.md)** — REST API endpoints
|
||||
- **[Provider Reference](../reference/PROVIDER_REFERENCE.md)** — All 226 providers
|
||||
- **[Provider Reference](../reference/PROVIDER_REFERENCE.md)** — All 338 providers
|
||||
- **[MCP Server](./MCP-SERVER.md)** — Model Context Protocol integration
|
||||
- **[A2A Server](./A2A-SERVER.md)** — Agent-to-Agent protocol
|
||||
- **[Cloud Agent](./CLOUD_AGENT.md)** — Cloud-based agents
|
||||
|
||||
@@ -22,7 +22,7 @@ When an IDE agent (e.g., GitHub Copilot, Cursor, Claude Code) makes an API call,
|
||||
|
||||
This means you can:
|
||||
|
||||
- **Reroute any agent to any provider**: Copilot talking to OpenAI? Redirect it to Anthropic Claude, Gemini, or any of OmniRoute's 226+ providers.
|
||||
- **Reroute any agent to any provider**: Copilot talking to OpenAI? Redirect it to Anthropic Claude, Gemini, or any of OmniRoute's 338 providers.
|
||||
- **Apply model mappings**: `gemini-3-flash` → `claude-sonnet-4.7` transparently at the handler level.
|
||||
- **Observe all agent traffic**: every intercepted request is published to the [Traffic Inspector](./TRAFFIC_INSPECTOR.md).
|
||||
- **Apply OmniRoute resilience**: combo routing, circuit breakers, fallbacks, and cost tracking work for IDE agent traffic too.
|
||||
|
||||
@@ -183,7 +183,7 @@ Both have "long-running tasks" but at different layers:
|
||||
curl http://localhost:20128/.well-known/agent.json
|
||||
```
|
||||
|
||||
Returns the Agent Card with all 5 skills, transports, and version.
|
||||
Returns the Agent Card with all 6 skills, transports, and version.
|
||||
|
||||
### Call OmniRoute as an A2A agent
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ lastUpdated: 2026-06-28
|
||||
|
||||
# OmniRoute MCP Server Documentation
|
||||
|
||||
> Model Context Protocol server with 104 tools across routing, cache, compression, memory, skills, proxy, pool, and context source operations.
|
||||
> Model Context Protocol server with 105 tools across routing, cache, compression, memory, skills, proxy, pool, and context source operations.
|
||||
>
|
||||
> Source of truth: `open-sse/mcp-server/server.ts` computes **104 unique tools** with `countUniqueMcpTools()`: 42 canonical definitions (including the six CCR lifecycle tools and the agent-skills trio), plus memory (3), skills (4), GitHub skills (3), pool (6), gamification (8), plugins (8), Notion (6), Obsidian (22), and two RTK-only compression tools.
|
||||
|
||||
@@ -369,7 +369,7 @@ MCP tool, prompt, and resource registries can compress descriptions at registrat
|
||||
|
||||
Description compression shrinks each tool's metadata; **tool-cardinality reduction** goes one step further by reducing _how many_ tools are announced at all. Advertising fewer tools in the `tools/list` manifest cuts the per-request token cost the client's model pays for the tool catalog ("layer 5" compression). The implementation is a pure, stateless filter in `open-sse/mcp-server/toolCardinality.ts` (`reduceToolManifest`), wired into the registration loop in `createMcpServer()` (`open-sse/mcp-server/server.ts`).
|
||||
|
||||
**Opt-in, off by default.** The filter only runs when at least one of two environment variables is set; with neither set, all 104 tools are announced unchanged.
|
||||
**Opt-in, off by default.** The filter only runs when at least one of two environment variables is set; with neither set, all 105 tools are announced unchanged.
|
||||
|
||||
| Variable | Mode |
|
||||
| :--------------- | :-------------------------------------------------------------------------------------- |
|
||||
|
||||
@@ -228,7 +228,7 @@ export async function handleComboChat(body, comboId): Promise<ChatResult> {
|
||||
}
|
||||
```
|
||||
|
||||
Supports **17 routing strategies** (see `src/shared/constants/routingStrategies.ts`):
|
||||
Supports **19 routing strategies** (see `src/shared/constants/routingStrategies.ts`):
|
||||
|
||||
| Strategy | Behavior |
|
||||
| ------------------- | ------------------------------------------------------------------------- |
|
||||
@@ -252,7 +252,7 @@ Supports **17 routing strategies** (see `src/shared/constants/routingStrategies.
|
||||
|
||||
### base.ts (1170 LOC)
|
||||
|
||||
The **abstract executor** that all 67 executors extend. It contains:
|
||||
The **abstract executor** that all 101 executors extend. It contains:
|
||||
|
||||
- `buildUrl()` — default URL construction (subclasses override for custom)
|
||||
- `buildHeaders()` — default headers (auth, content-type)
|
||||
@@ -368,7 +368,7 @@ const result = await executor.execute({
|
||||
});
|
||||
````
|
||||
|
||||
The factory is generated from `config/providerRegistry.ts` which lists all 212+ providers and their executor class.
|
||||
The factory is generated from `config/providerRegistry.ts` which lists all 338 providers and their executor class.
|
||||
|
||||
---
|
||||
|
||||
@@ -408,7 +408,7 @@ Common translations:
|
||||
|
||||
- **30+ tools** (provider management, combos, memory, cache, compression, 1proxy, skills)
|
||||
- **3 transports**: stdio, SSE, Streamable HTTP
|
||||
- **13 scopes** for fine-grained authorization
|
||||
- **31 scopes** for fine-grained authorization
|
||||
|
||||
### Tool Registration
|
||||
|
||||
@@ -487,7 +487,7 @@ This handles:
|
||||
|
||||
| File | Purpose |
|
||||
| ----------------------------- | --------------------------------- |
|
||||
| `providerRegistry.ts` | 212+ provider definitions |
|
||||
| `providerRegistry.ts` | 338 provider definitions |
|
||||
| `providerModels.ts` | Model aliases, format mapping |
|
||||
| `constants.ts` | Timeouts, limits, status codes |
|
||||
| `defaultThinkingSignature.ts` | Default Claude thinking signature |
|
||||
|
||||
@@ -88,9 +88,9 @@ README variants) are not yet handled by the new pipeline and are still used.
|
||||
### Source of Truth
|
||||
|
||||
- **UI strings**: `src/i18n/messages/en.json` (English source, ~2800 keys)
|
||||
- **Locale files**: `src/i18n/messages/{locale}.json` (30 translations)
|
||||
- **Locale files**: `src/i18n/messages/{locale}.json` (43 translations)
|
||||
- **Framework**: `next-intl` with cookie-based locale resolution
|
||||
- **Config**: `src/i18n/config.ts` — defines all 30 locales, language names, flags
|
||||
- **Config**: `src/i18n/config.ts` — defines all 43 locales, language names, flags
|
||||
|
||||
### Runtime Flow
|
||||
|
||||
@@ -304,7 +304,7 @@ invocation. Use `config lang set` to persist.
|
||||
|
||||
43 locale files ship in `bin/cli/locales/`. Full translations: `en`, `pt-BR`.
|
||||
Scaffold-only (all keys fall back to `en`): `bn`, `gu`, `he`, `in`, `mr`, `ms`, `phi`, `sw`, `ta`, `te`, `ur`.
|
||||
All other 29 locales have `common` + `program` keys translated.
|
||||
All other 30 locales have `common` + `program` keys translated.
|
||||
|
||||
### Adding a new CLI locale
|
||||
|
||||
@@ -533,7 +533,7 @@ python3 scripts/i18n/validate_translation.py quick -l '${{ matrix.lang }}'
|
||||
|
||||
```
|
||||
src/i18n/
|
||||
├── config.ts # Locale definitions (30 locales, RTL config)
|
||||
├── config.ts # Locale definitions (43 locales, RTL config)
|
||||
├── request.ts # Runtime locale resolution
|
||||
└── messages/
|
||||
├── en.json # Source of truth (~2800 keys)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -1,20 +1,21 @@
|
||||
---
|
||||
title: "Provider Reference"
|
||||
version: 3.8.50
|
||||
lastUpdated: 2026-08-05
|
||||
lastUpdated: 2026-08-12
|
||||
---
|
||||
|
||||
# Provider Reference
|
||||
|
||||
> **Auto-generated** from `src/shared/constants/providers.ts` — do not edit by hand.
|
||||
> Regenerate with: `npm run gen:provider-reference`
|
||||
> **Last generated:** 2026-08-05
|
||||
> **Last generated:** 2026-08-12
|
||||
|
||||
Total providers: **291**. See category breakdown below.
|
||||
Total providers: **338**. See category breakdown below.
|
||||
|
||||
## Categories
|
||||
|
||||
- **Free** — free tier with API key (configured via dashboard)
|
||||
- **No-auth** — public endpoints that require no key or sign-in at all
|
||||
- **OAuth** — sign-in flow handled by OmniRoute, no API key needed
|
||||
- **Web cookie** — wraps the provider's web app via cookie auth
|
||||
- **API key** — paid provider configured via API key (free credits may apply)
|
||||
@@ -33,7 +34,22 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
|
||||
|
||||
---
|
||||
|
||||
## OAuth Providers (23)
|
||||
## No-auth Providers (no key required) (10)
|
||||
|
||||
| ID | Alias | Name | Tags | Website | Notes | Tool calling |
|
||||
|----|-------|------|------|---------|-------|--------------|
|
||||
| `aihorde` | `horde` | AI Horde | No-auth | [link](https://aihorde.net) | No API key required — uses AI Horde's documented anonymous key. Adding a free aihorde.net key is optional and only buys higher queue priority (kudos). | — |
|
||||
| `auggie` | `aug` | Augment (Auggie CLI) | No-auth | [link](https://augmentcode.com) | No API key stored by OmniRoute. Install the Auggie CLI and run `auggie login` on this machine, then OmniRoute spawns it locally for each request. | — |
|
||||
| `chipotle` | `pepper` | Chipotle Pepper AI (Free) | No-auth | [link](https://amelia.chipotle.com) | No credentials required. Uses Chipotle's public support chatbot via reverse-engineered SockJS/STOMP protocol. | — |
|
||||
| `devin-cli-agentic` | `dva` | Devin CLI Agentic Bridge | No-auth | [link](https://docs.devin.ai/work-with-devin/devin-cli) | Authentication is owned by the official Devin CLI in its isolated bridge volume. | emulated |
|
||||
| `duckduckgo-web` | `ddgw` | DuckDuckGo AI Chat | No-auth | [link](https://duckduckgo.com/duckchat) | No credentials required — DuckDuckGo AI Chat is anonymous and free. | emulated |
|
||||
| `felo-web` | `felo` | Felo | No-auth | [link](https://felo.ai) | No credentials required — Felo is a free, no-signup chat/search aggregator. | — |
|
||||
| `mimocode` | `mcode` | MiMoCode (Free) | No-auth | [link](https://mimo.mi.com) | No API key required. The executor auto-generates JWT tokens via device fingerprint bootstrap. | — |
|
||||
| `opencode` | `oc` | OpenCode Free | No-auth | [link](https://opencode.ai) | No API key required — uses OpenCode's public free endpoint. | — |
|
||||
| `theoldllm` | `tllm` | The Old LLM (Free) | No-auth | [link](https://theoldllm.vercel.app) | No credentials required. The executor auto-generates access tokens via an embedded Playwright browser instance. | — |
|
||||
| `veoaifree-web` | `veo-free` | Veo AI Free | No-auth, video | [link](https://veoaifree.com) | No auth required. Rate limited to 6 requests/hour per IP. | — |
|
||||
|
||||
## OAuth Providers (25)
|
||||
|
||||
| ID | Alias | Name | Tags | Website | Notes |
|
||||
|----|-------|------|------|---------|-------|
|
||||
@@ -46,7 +62,7 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
|
||||
| `codebuddy-cn` | `cbcn` | CodeBuddy CN | OAuth | [link](https://copilot.tencent.com) | Tencent CodeBuddy CN (copilot.tencent.com). Sign in via the official CLI device-code flow, or paste a direct API key (sent as Authorization: Bearer). Catalog: GLM / Kimi / MiniMax / DeepSeek / Hunyuan. |
|
||||
| `codex` | `cx` | OpenAI Codex | OAuth | — | — |
|
||||
| `cursor` | `cu` | Cursor IDE | OAuth | — | — |
|
||||
| `devin-cli` | `dv` | Devin CLI (Official) | OAuth | [link](https://cli.devin.ai) | Requires the Devin CLI binary. Run `devin auth login` to authenticate, or provide your WINDSURF_API_KEY. Install: https://cli.devin.ai |
|
||||
| `devin-cli` | `dv` | Devin CLI | OAuth | [link](https://cli.devin.ai) | Requires the Devin CLI binary. Run `devin auth login` to authenticate, or provide your WINDSURF_API_KEY. Install: https://cli.devin.ai |
|
||||
| `devin-desktop` | — | Devin Desktop | OAuth | [link](https://devin.ai) | Paste an existing Devin API key from an authenticated Devin session. Key export availability and steps vary by Devin version and account. |
|
||||
| `ghe-copilot` | `ghe-copilot` | GitHub Enterprise Copilot | OAuth | — | Enter your GHE instance URL (e.g., https://ghe.company.com) in provider settings, then authenticate via device flow. |
|
||||
| `github` | `gh` | GitHub Copilot | OAuth | — | — |
|
||||
@@ -55,13 +71,15 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
|
||||
| `kilocode` | `kc` | Kilo Code | OAuth | — | — |
|
||||
| `kimi-coding` | `kmc` | Kimi Code CLI | OAuth | [link](https://www.kimi.com/code?aff=omniroute) | Sign in with the same Kimi account used by Kimi Code CLI. OmniRoute uses the CLI OAuth flow and Kimi Coding Plan endpoints. |
|
||||
| `kiro` | `kr` | Kiro AI | OAuth | — | Free tier: 50 credits/month (~25K–100K tokens). ⚠️ Kiro ToS prohibits third-party proxy/harness use. |
|
||||
| `openference` | `of` | Openference | OAuth | [link](https://openference.com) | Sign in with your Openference account to route requests through api.openference.com. An active plan is required for inference — OAuth may authenticate but return 402 without one. |
|
||||
| `qoder` | `if` | Qoder | OAuth | — | — |
|
||||
| `raycast` | `rc` | Raycast Pro AI | OAuth | [link](https://raycast.com/ai) | Unofficial integration — uses your Raycast Pro subscription via credentials from the macOS app (Auto-Import or manual capture). May break on Raycast updates. Not for redistribution; personal use only. |
|
||||
| `trae` | `tr` | Trae | OAuth | [link](https://trae.ai) | Trae is an AI-native IDE by ByteDance (SOLO remote agent). Authorize via trae.ai in the popup, or sign in at solo.trae.ai and paste the Cloud-IDE-JWT (sent as 'Authorization: Cloud-IDE-JWT <token>', ~14-day lifetime) as the access token; web_id/biz_user_id/user_unique_id/scope/tenant/region propagate via providerSpecificData. No headless refresh for pasted tokens — re-paste on expiry. |
|
||||
| `xai-oauth` | `xao` | xAI OAuth (Grok) | OAuth | [link](https://x.ai) | Sign in with xAI to use api.x.ai models such as Grok 4.5. This is separate from Grok Build JWT sessions, which use cli-chat-proxy.grok.com and grok-build model aliases. |
|
||||
| `zed` | `zd` | Zed IDE | OAuth | [link](https://zed.dev) | Zed stores LLM provider credentials (OpenAI, Anthropic, Google, Mistral, xAI) in the OS keychain. Use the Import button below to discover and import them automatically. |
|
||||
| `zed-hosted` | — | Zed Hosted Models | OAuth | [link](https://zed.dev) | Sign in with your Zed account (native-app sign-in). OmniRoute generates a one-time RSA keypair and opens zed.dev to authorize it — on a remote/headless install, copy the resulting 127.0.0.1 callback URL from your browser's address bar and paste it back here. Distinct from the 'Zed IDE' credential-import entry above: this proxies chat completions through Zed's own hosted model aggregator (cloud.zed.dev), fronting Anthropic/OpenAI/Google/xAI models under your Zed plan. |
|
||||
|
||||
## Web Cookie Providers (31)
|
||||
## Web Cookie Providers (34)
|
||||
|
||||
| ID | Alias | Name | Tags | Website | Notes | Tool calling |
|
||||
|----|-------|------|------|---------|-------|--------------|
|
||||
@@ -69,9 +87,11 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
|
||||
| `adobe-firefly` | `firefly` | Adobe Firefly (Image/Video) | Web cookie | [link](https://firefly.adobe.com) | RECOMMENDED: firefly.adobe.com signed-in → F12 → Network → click firefly-3p.ff.adobe.io (generate-async or models/discovery) → Request Headers → Authorization → copy the token AFTER 'Bearer ' (starts with eyJ…). Cookie-only from firefly.adobe.com mints a GUEST token → 401/403; only multi-domain IMS cookies (adobelogin.com) or that Bearer JWT work. Unofficial/experimental media + Limits. | — |
|
||||
| `blackbox-web` | `bb-web` | Blackbox Web (Subscription) | Web cookie | [link](https://app.blackbox.ai) | Paste your __Secure-authjs.session-token value or full cookie header from app.blackbox.ai | emulated |
|
||||
| `chatgpt-web` | `cgpt-web` | ChatGPT Web (Plus/Pro) | Web cookie | [link](https://chatgpt.com) | Paste your __Secure-next-auth.session-token cookie value from chatgpt.com | emulated |
|
||||
| `chatgpt-web-codex` | `cgpt-codex` | ChatGPT Web (Codex) | Web cookie | [link](https://chatgpt.com) | Paste the full ChatGPT Cookie header. OmniRoute verifies it in an isolated headless browser profile. | native |
|
||||
| `claude-web` | `cw` | Claude Web | Web cookie | [link](https://claude.ai) | Paste your session cookie from claude.ai | none |
|
||||
| `conol-web` | `cnl` | Conol (Unofficial/Experimental) | Web cookie | [link](https://conol.ai) | Use browser sign-in, or paste the full Cookie header from conol.ai. The __Secure-better-auth.session_token cookie is required. | — |
|
||||
| `copilot-m365-web` | `m365copilot` | Microsoft 365 Copilot (BizChat) | Web cookie | [link](https://m365.cloud.microsoft/chat) | Sign in at m365.cloud.microsoft/chat, then open DevTools → Network → filter 'WS' → click the Chathub WebSocket connection. Copy both the access_token query parameter AND the account-specific Chathub path segment from its request URL (wss://…/Chathub/<path>?…&access_token=…). It is NOT an Authorization: Bearer header on an XHR/Fetch request. The token is short-lived; this is an unofficial integration. | — |
|
||||
| `copilot-web` | `copilot` | Microsoft Copilot Web | Web cookie | [link](https://copilot.microsoft.com) | Paste your access_token from copilot.microsoft.com (or export a .har file from DevTools while logged in) | — |
|
||||
| `copilot-web` | `copilot` | Microsoft Copilot Web | Web cookie | [link](https://copilot.microsoft.com) | Paste the access_token from an authenticated copilot.microsoft.com request (DevTools → Network → Authorization), or export a HAR while logged in | — |
|
||||
| `deepseek-web` | `ds-web` | DeepSeek Web | Web cookie | [link](https://chat.deepseek.com) | Paste your userToken from chat.deepseek.com — DevTools → Application → Local Storage → userToken | emulated |
|
||||
| `doubao-web` | `db` | Dola Web (ByteDance) | Web cookie | [link](https://www.dola.com) | Paste the full Cookie header from www.dola.com. It should include sessionid, ttwid, and s_v_web_id. If s_v_web_id is unavailable, fp=verify_... from a chat/completion request URL can be used as a fallback. | — |
|
||||
| `gemini-business` | `gembiz` | Gemini Business (Enterprise) | Web cookie | [link](https://business.gemini.google) | From your enterprise account: open business.gemini.google/home/cid/{your-cid}, then copy __Secure-1PSID and __Secure-1PSIDTS cookies from DevTools → Application → Cookies. Paste as a cookie header below. | — |
|
||||
@@ -91,19 +111,20 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
|
||||
| `promptql` | `pql` | PromptQL (Unofficial/Experimental) | Web cookie | [link](https://prompt.ql.app) | Paste the Bearer JWT from prompt.ql.app DevTools → Network → graphql → Authorization (token only). Optional projectId + session Cookie for refresh. | — |
|
||||
| `qwen-web` | `qwen-web` | Qwen Web (Free) | Web cookie | [link](https://chat.qwen.ai) | Open chat.qwen.ai, log in, then open DevTools → Application → Local Storage → copy the "token" value (or use tongyi_sso_ticket cookie as Bearer token). | emulated |
|
||||
| `t3-web` | `t3chat` | t3.chat (Pro/Free) | Web cookie | [link](https://t3.chat) | Open t3.chat in your browser, log in, then open DevTools → Application → Local Storage → https://t3.chat. Copy the value of 'convex-session-id'. Also open DevTools → Network, copy the Cookie header from any request. Paste both values here. See provider setup docs for a step-by-step guide. | emulated |
|
||||
| `tinycms-web` | `tcw` | TinyCMS Web (Free/Sub) | Web cookie | [link](https://site.tinycms.xyz) | Go to site.tinycms.xyz, open DevTools → Application → Local Storage, copy the value of 'app-config-uuid' (starts with 'R'), and paste it here. | — |
|
||||
| `v0-vercel-web` | `v0-vercel-web` | v0 Vercel Web (Code Gen) | Web cookie | [link](https://v0.dev) | Paste your session cookie from v0.dev (DevTools → Application → Cookies) | — |
|
||||
| `venice-web` | `ven` | Venice Web (Privacy) | Web cookie | [link](https://venice.ai) | Paste your session cookie from venice.ai (DevTools → Application → Cookies) | — |
|
||||
| `yuanbao-web` | `ybw` | Tencent Yuanbao (Free) | Web cookie | [link](https://yuanbao.tencent.com) | Log in to yuanbao.tencent.com, then paste the full Cookie header (DevTools → Network → any /api request → Request Headers → Cookie). It must contain hy_user and hy_token. | — |
|
||||
| `zai-web` | `zw` | Z.ai Web (Free) | Web cookie | [link](https://chat.z.ai) | Paste the full Cookie header from chat.z.ai (must include the token=<JWT> cookie) | — |
|
||||
| `zai-web` | `zw` | Z.ai Web | Web cookie | [link](https://chat.z.ai) | Copy the "token" value from chat.z.ai → DevTools → Application → Local Storage. Do not copy cookies; OmniRoute handles the per-request CAPTCHA through its browser transport. | — |
|
||||
| `zenmux-free` | `zmf` | ZenMux Free (Web) | Web cookie | [link](https://zenmux.ai) | Login at zenmux.ai, then export all cookies using EditThisCookie or Cookie-Editor and paste the full Cookie header string here. Refresh every ~30 days. | — |
|
||||
|
||||
## API Key Providers (paid / paid-with-free-credits) (196)
|
||||
## API Key Providers (paid / paid-with-free-credits) (227)
|
||||
|
||||
| ID | Alias | Name | Tags | Website | Notes |
|
||||
|----|-------|------|------|---------|-------|
|
||||
| `360ai` | `360ai` | 360 AI | API key | [link](https://ai.360.cn) | Get API key at ai.360.cn |
|
||||
| `agentrouter` | `agentrouter` | AgentRouter | API key, aggregator | [link](https://agentrouter.org) | $200 free credits on signup - multi-model routing gateway |
|
||||
| `agnes` | `agnes` | Agnes AI | API key | [link](https://agnes-ai.com) | Get API key at agnes-ai.com |
|
||||
| `agnes` | `agnes` | Agnes AI | API key, video | [link](https://agnes-ai.com) | Get API key at agnes-ai.com |
|
||||
| `ai21` | `ai21` | AI21 Labs | API key | [link](https://www.ai21.com) | $10 trial credits on signup (valid 3 months), no credit card required |
|
||||
| `aimlapi` | `aiml` | AI/ML API | API key, aggregator | [link](https://aimlapi.com) | Free tier paused (2026) — AI/ML API is now pay-as-you-go only (min $20 top-up); no recurring free credits. |
|
||||
| `ainative` | `ainative` | AINative Studio | API key | [link](https://ainative.studio) | Create a free API key at ainative.studio (no card), then paste it here as a Bearer token. |
|
||||
@@ -112,8 +133,10 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
|
||||
| `alibaba-cn` | `ali-cn` | Alibaba (China) | API key | [link](https://dashscope.console.aliyun.com/) | — |
|
||||
| `ant-ling` | `ling` | Ant Ling / Ring (inclusionAI) | API key | [link](https://developer.ant-ling.com/en/docs/) | Register and create an API key at the Ant Ling API console (https://chat.ant-ling.com/open), then paste it here. OmniRoute routes chat traffic to https://api.ant-ling.com/v1/chat/completions; the provider is OpenAI-compatible and also exposes an Anthropic-compatible surface. |
|
||||
| `anthropic` | `anthropic` | Anthropic | API key | [link](https://platform.claude.com) | — |
|
||||
| `anyapi` | `anyapi` | AnyAPI AI | API key, aggregator | [link](https://anyapi.ai) | Free plan: 100,000 ANY Tokens/day and 100 RPM for eligible Free/Basic models; no credit card required. |
|
||||
| `api-airforce` | `af` | Api.airforce | API key | [link](https://api.airforce) | 55 free tier models including Grok-3, Claude 3.7, Qwen3, Kimi-K2, Gemini 2.5 Flash, DeepSeek-V3 |
|
||||
| `arcee-ai` | `arcee` | Arcee AI | API key | [link](https://arcee.ai) | Get API key at arcee.ai |
|
||||
| `auriko` | `auriko` | Auriko | API key, aggregator | [link](https://www.auriko.ai) | Free plan publishes 1,000 Platform RPM and 10,000 BYOK RPM. Platform inference still passes through provider cost; this is not a free-token pool or unlimited free inference. |
|
||||
| `azure-ai` | `azure-ai` | Azure AI Foundry | API key, enterprise | [link](https://learn.microsoft.com/azure/ai-foundry) | Use your Azure AI Foundry key. Base URL can be https://<resource>.services.ai.azure.com/openai/v1/ or https://<resource>.openai.azure.com/openai/v1/. |
|
||||
| `azure-openai` | `azure` | Azure OpenAI | API key, enterprise | [link](https://azure.microsoft.com/products/ai-services/openai-service) | Use your Azure OpenAI API key. Base URL should be your resource endpoint, for example https://my-resource.openai.azure.com. |
|
||||
| `bai` | `bai` | b.ai | API key | [link](https://b.ai) | Bearer API key for the b.ai OpenAI-compatible LLM gateway (distinct from TheB.AI). Create a key at https://docs.b.ai, then use https://api.b.ai/v1 as the OpenAI-compatible base URL. |
|
||||
@@ -130,10 +153,13 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
|
||||
| `bytez` | `bytez` | Bytez | API key | [link](https://bytez.com) | $1 free credits, refreshes every 4 weeks |
|
||||
| `cerebras` | `cerebras` | Cerebras | API key | [link](https://inference.cerebras.ai) | Free Trial: 1M tokens/day, 30K TPM, 5 RPM — no credit card. |
|
||||
| `charm-hyper` | `charm-hyper` | Charm Hyper | API key | [link](https://hyper.charm.land) | 100 free monthly Hypercredits on signup |
|
||||
| `chat-oripe` | `chat-oripe` | Chat Oripe | API key, aggregator | [link](https://api.oriper.com) | Official metadata advertises 2M tokens/month, but the public site and documentation were blocked during audit; treat the quota and brand mapping as unconfirmed. |
|
||||
| `chatanywhere` | `chatanywhere` | ChatAnywhere | API key, aggregator | [link](https://chatanywhere.tech) | Personal, educational or research use only: public documentation cites 10,000 points/day and 200 requests/day per IP/key; do not use for commercial traffic. |
|
||||
| `cheaperinference` | `cinf` | Cheaper Inference | API key | [link](https://cheaperinference.com/?utm_source=omniroute) | — |
|
||||
| `chenzk` | `chenzk` | Chenzk API | API key | [link](https://chenzk.top) | — |
|
||||
| `chutes` | `chutes` | Chutes.ai | API key, aggregator | [link](https://chutes.ai) | Bearer API key for the Chutes OpenAI-compatible gateway. |
|
||||
| `clarifai` | `clarifai` | Clarifai | API key, enterprise | [link](https://docs.clarifai.com) | Use your Clarifai PAT or app-specific API key. OmniRoute targets the OpenAI-compatible endpoint at https://api.clarifai.com/v2/ext/openai/v1 and authenticates with Authorization: Key <token>. |
|
||||
| `cloudcode-one` | `cloudcode-one` | CloudCode.ONE | API key, aggregator | [link](https://cloudcode.one) | Published free models include glm-4.7-flash and glm-4.6v-flash; no numeric quota is published, and key creation may require credit or a coupon. |
|
||||
| `cloudflare-ai` | `cf` | Cloudflare Workers AI | API key | [link](https://developers.cloudflare.com/workers-ai) | Requires API Token AND Account ID (found at dash.cloudflare.com) |
|
||||
| `clova-studio` | `clova` | Naver CLOVA Studio | API key | [link](https://api.ncloud-docs.com/docs/en/ai-naver-clovastudio-summary) | — |
|
||||
| `codestral` | `codestral` | Codestral | API key | [link](https://mistral.ai) | — |
|
||||
@@ -141,9 +167,10 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
|
||||
| `command-code` | `cmd` | Command Code | API key | [link](https://commandcode.ai/) | Use a Command Code API key. Requests are sent to Command Code's /alpha/generate endpoint. |
|
||||
| `coze` | `coze` | Coze | API key | [link](https://coze.com) | Get API key at coze.com/open/api |
|
||||
| `crof` | `crof` | CrofAI | API key | [link](https://crof.ai) | — |
|
||||
| `dahl` | `dahl` | Dahl | API key | [link](https://inference.dahl.global) | Click 'Add Account' to auto-generate a token. |
|
||||
| `dahl` | `dahl` | Dahl | API key | [link](https://inference.dahl.global) | Click 'Add Account' to auto-generate a token, or add a manual API key. |
|
||||
| `databricks` | `databricks` | Databricks | API key, enterprise | [link](https://www.databricks.com) | — |
|
||||
| `datarobot` | `datarobot` | DataRobot | API key, enterprise | [link](https://docs.datarobot.com) | Use your DataRobot API token. Optional Base URL can be the account root (for LLM Gateway) or a deployment URL under /api/v2/deployments/<id>. |
|
||||
| `deepai` | `deepai` | DeepAI | API key, image | [link](https://deepai.org) | Use your DeepAI API key. Get one at deepai.org — requires a Pro subscription ($9.99/mo). |
|
||||
| `deepinfra` | `deepinfra` | DeepInfra | API key | [link](https://deepinfra.com) | Free signup credits for API testing and model exploration |
|
||||
| `deepseek` | `ds` | DeepSeek | API key | [link](https://platform.deepseek.com) | 5M free tokens on signup - no credit card required |
|
||||
| `dgrid` | `dgrid` | DGrid | API key | [link](https://dgrid.ai) | DGrid Free Models Router: 10 requests/minute and 100 requests/day. A $5 lifetime top-up unlocks up to 20 requests/minute and 1,000 requests/day. |
|
||||
@@ -151,13 +178,18 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
|
||||
| `digitalocean` | `digitalocean` | DigitalOcean | API key | [link](https://docs.digitalocean.com/products/ai-platform/) | — |
|
||||
| `dit` | `dai` | DIT.ai | API key | [link](https://dit.ai) | Use your dit.ai API key in Authorization: Bearer <key>. Fully OpenAI-compatible — a drop-in replacement, just change the base URL to https://api.dit.ai/v1. |
|
||||
| `doubao` | `doubao` | Doubao | API key | [link](https://doubao.com) | Get API key at console.volcengine.com |
|
||||
| `dxnt` | `dxnt` | DXNT / DX Token | API key, aggregator | [link](https://www.dxnt.com) | Free accounts are documented at 100 calls/day; the quota may increase through invitations and can vary by account. |
|
||||
| `electronhub` | `electronhub` | Electron Hub | API key, aggregator | [link](https://www.electronhub.ai) | Free plan: 5 RPM, $0.25 weekly credits and 10 Neutrinos/day for :free models; family budgets also apply. |
|
||||
| `empower` | `empower` | Empower | API key, aggregator | [link](https://docs.empower.dev) | Bearer API key for the Empower OpenAI-compatible endpoint. |
|
||||
| `factory` | `factory` | Factory | API key | [link](https://factory.ai) | Bearer API key for the Factory OpenAI-compatible gateway. |
|
||||
| `fal-ai` | `fal` | Fal.ai | API key, image | [link](https://fal.ai) | — |
|
||||
| `fastrouter` | `fastrouter` | FastRouter | API key, aggregator | [link](https://fastrouter.ai) | Models with the :free suffix allow 10 requests/day per organization and model; availability may change. |
|
||||
| `featherless-ai` | `featherless` | Featherless AI | API key | [link](https://featherless.ai) | Free tier available — no credit card required |
|
||||
| `fenayai` | `fenayai` | FenayAI | API key, aggregator | [link](https://fenayai.com) | Bearer API key for the FenayAI OpenAI-compatible gateway. |
|
||||
| `fireworks` | `fireworks` | Fireworks AI | API key | [link](https://fireworks.ai) | $1 free starter credits on signup for API testing |
|
||||
| `free-ai` | `free-ai` | Free.ai | API key, aggregator | [link](https://free.ai) | 30,000 tokens/day cover self-hosted models after email verification. Usage beyond the pool can bill at raw cost, and premium external models are paid. |
|
||||
| `freeaiapikey` | `faik` | FreeAIAPIKey | API key | [link](https://freeaiapikey.com) | — |
|
||||
| `freeinference` | `freeinference` | FreeInference | API key, aggregator | [link](https://freeinference.org) | Free research access without a card; non-Harvard applicants require manual approval and no numeric quota is publicly guaranteed. |
|
||||
| `freemodel-dev` | `fmd` | FreeModel.dev | API key | [link](https://freemodel.dev) | $300 free credits on signup — no credit card required. Access GPT-5.4 and GPT-5.5 (OpenAI's latest flagship models) through an OpenAI-compatible API. |
|
||||
| `freepik` | `fpk` | Freepik (Mystic) | API key, image | [link](https://freepik.com) | Get API key at freepik.com/developers (Mystic image endpoint) |
|
||||
| `freetheai` | `fta` | FreeTheAi | API key, aggregator | [link](https://freetheai.xyz) | Join the FreeTheAi Discord to get your free API key. |
|
||||
@@ -171,7 +203,6 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
|
||||
| `gemini` | `gemini` | Gemini (Google AI Studio) | API key | [link](https://aistudio.google.com) | Free forever: 1,500 req/day for Gemini 2.5 Flash — no credit card, get key at aistudio.google.com |
|
||||
| `getgoapi` | `ggo` | GoAPI | API key, aggregator | [link](https://api.getgoapi.com) | — |
|
||||
| `gigachat` | `gigachat` | GigaChat (Sber) | API key | [link](https://developers.sber.ru) | — |
|
||||
| `github-models` | `ghm` | GitHub Models | API key | [link](https://github.com/marketplace/models) | Create a GitHub PAT with 'models: read' scope at github.com/settings/tokens |
|
||||
| `gitlab` | `gitlab` | GitLab Duo PAT | API key | [link](https://docs.gitlab.com/user/duo_agent_platform/code_suggestions/) | GitLab personal access token for the public Code Suggestions API. Configure a self-hosted base URL when not using gitlab.com. |
|
||||
| `gitlawb` | `glb` | Gitlawb Opengateway (MiMo) | API key | [link](https://opengateway.gitlawb.com) | Free MiMo (xiaomi/mimo-v2.5) revoked 2026-05 — Opengateway is now a pay-as-you-go credit gateway; no recurring free model. |
|
||||
| `gitlawb-gmi` | `glb-gmi` | Gitlawb Opengateway (GMI Cloud) | API key | [link](https://opengateway.gitlawb.com) | Free Nemotron promo ended 2026-06 — the GMI Cloud route is now pay-as-you-go credit only. |
|
||||
@@ -182,6 +213,7 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
|
||||
| `hackclub` | `hc` | Hackclub AI | API key, aggregator | [link](https://ai.hackclub.com) | Sign in with your Hack Club account at ai.hackclub.com. |
|
||||
| `haiper` | `hp` | Haiper | API key, video | [link](https://haiper.ai) | Get API key at haiper.ai/haiper-api |
|
||||
| `hcnsec` | `hcnsec` | Huancheng Public API | API key | [link](https://api.hcnsec.cn) | Get API key at api.hcnsec.cn |
|
||||
| `helyxai` | `helyxai` | Helyx AI | API key, aggregator | [link](https://helyxai.space) | Operational Free plan documents 100,000 tokens/day; the site's separate 2M+ marketing claim conflicts and is not treated as a quota guarantee. |
|
||||
| `heroku` | `heroku` | Heroku AI | API key, enterprise | [link](https://www.heroku.com) | — |
|
||||
| `huggingface` | `hf` | HuggingFace | API key | [link](https://huggingface.co) | Free Inference API for thousands of models (Whisper, VITS, SDXL…) |
|
||||
| `hyperbolic` | `hyp` | Hyperbolic | API key | [link](https://hyperbolic.xyz) | $1-5 trial credits on signup for serverless inference |
|
||||
@@ -201,20 +233,29 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
|
||||
| `laozhang` | `lz` | LaoZhang AI | API key, aggregator | [link](https://api.laozhang.ai) | — |
|
||||
| `leonardo` | `leo` | Leonardo AI | API key, video | [link](https://leonardo.ai) | Get API key at leonardo.ai/developer |
|
||||
| `liquid` | `liquid` | Liquid AI | API key | [link](https://liquid.ai) | Get API key at liquid.ai |
|
||||
| `literouter` | `literouter` | LiteRouter | API key, aggregator | [link](https://literouter.com) | Free model variants use the :free suffix; daily credit limits vary by model and free input is capped at 5,000 tokens. |
|
||||
| `llamagate` | `llamagate` | LlamaGate | API key | [link](https://llamagate.ai) | — |
|
||||
| `llm7` | `llm7` | LLM7.io | API key | [link](https://llm7.io) | No signup required - 2 req/s, 20 RPM, 100 req/hr free tier |
|
||||
| `llm-kiwi` | `llmkiwi` | LLM.Kiwi | API key, aggregator | [link](https://llm.kiwi) | Free plan exposes auto and hrLLM; the published 40 requests/hour limit applies to hrLLM. |
|
||||
| `llm7` | `llm7` | LLM7.io | API key | [link](https://llm7.io) | Use any non-empty key (for example 'unused'). If older built-in models return model_unavailable, use Available Models → Import from /models or Auto-Sync; verified live model: gemini-3.1-flash-lite. |
|
||||
| `llmgateway` | `llmgateway` | LLM Gateway | API key, aggregator | [link](https://llmgateway.io) | Hosted Free plan: free-priced models are limited to 5 requests per 10 minutes when the account has no credits. |
|
||||
| `longcat` | `lc` | LongCat AI | API key | [link](https://longcat.chat/platform/docs) | Free: one-time 10M-token grant after account signup + KYC verification (LongCat-2.0). One-time only — not a recurring daily/monthly allowance. |
|
||||
| `maritalk` | `maritalk` | Maritalk | API key | [link](https://www.maritaca.ai) | — |
|
||||
| `meganova-ai` | `meganova-ai` | MegaNova AI | API key, aggregator | [link](https://meganova.ai) | Free signup without a card. Published Tier 1 per-model quotas total 550 requests/day; they are not a shared global pool, and paid overage can apply if enabled. |
|
||||
| `meta-llama` | `meta` | Meta Llama API | API key | [link](https://llama.developer.meta.com) | — |
|
||||
| `minimax` | `minimax` | Minimax Coding | API key, video | [link](https://www.minimax.io) | — |
|
||||
| `minimax-cn` | `minimax-cn` | Minimax (China) | API key | [link](https://www.minimaxi.com) | — |
|
||||
| `mistral` | `mistral` | Mistral | API key | [link](https://mistral.ai) | Free Experiment tier: rate-limited access to all models, no credit card required |
|
||||
| `mixedbread` | `mxbai` | Mixedbread AI | API key | [link](https://www.mixedbread.com) | Bearer API key for the Mixedbread embeddings API. |
|
||||
| `mixlayer` | `mixlayer` | Mixlayer | API key, aggregator | [link](https://www.mixlayer.com) | The qwen/qwen3.5-4b-free model is free for prototyping and rate-limited; no fixed public RPM or daily quota is confirmed. |
|
||||
| `mnn-ai` | `mnn-ai` | MNN AI | API key, aggregator | [link](https://mnnai.ru) | Free plan: $1 monthly credits, 10 RPM and access only to models marked Free. |
|
||||
| `modal` | `mdl` | Modal | API key, enterprise | [link](https://modal.com/docs) | Use the bearer token that protects your Modal deployment, if enabled. Base URL should point to your OpenAI-compatible Modal app, for example https://<workspace>--<app>.modal.run/v1. |
|
||||
| `modelscope` | `ms` | ModelScope | API key | [link](https://modelscope.cn) | Free tier via ModelScope API-Inference — Alibaba account required. |
|
||||
| `monsterapi` | `monster` | MonsterAPI | API key | [link](https://monsterapi.ai) | Get API key at monsterapi.ai |
|
||||
| `moonshot` | `moonshot` | Kimi | API key | [link](https://platform.kimi.ai?aff=omniroute) | — |
|
||||
| `morph` | `morph` | Morph | API key | [link](https://morphllm.com) | Free tier: 250K credits/month, $0 |
|
||||
| `muse-code` | `mc` | Muse Code (Meta) | API key | [link](https://github.com/meta-llama/llama-stack) | Use your META_API_KEY env var as a Bearer token. Muse Code CLI uses the OpenAI Responses API wire format (POST /responses). |
|
||||
| `naga-ac` | `naga` | Naga.ac | API key, aggregator | [link](https://naga.ac) | Get API key at naga.ac — Google/GitHub/Discord signup available. |
|
||||
| `naga-ai` | `naga-ai` | Naga AI | API key, aggregator | [link](https://naga.ac) | Models marked :free are publicly listed, but no numeric quota is confirmed. Naga's policy warns that free-tier prompts and outputs may be collected or used for training. |
|
||||
| `nanogpt` | `nanogpt` | NanoGPT | API key | [link](https://nano-gpt.com) | — |
|
||||
| `nara` | `nara` | NaraRouter | API key | [link](https://bynara.id) | Get a free API key via NaraRouter's Telegram channel, then paste it here as a Bearer token. |
|
||||
| `navy` | `navy` | NavyAI | API key | [link](https://api.navy) | Create a free API key from the NavyAI dashboard, then paste it here as a Bearer token. |
|
||||
@@ -227,11 +268,13 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
|
||||
| `nube` | `nube` | Nube.sh | API key | [link](https://nube.sh) | — |
|
||||
| `nvidia` | `nvidia` | NVIDIA NIM | API key | [link](https://build.nvidia.com) | Free dev access: ~40 RPM, 70+ models (Kimi K2.5, GLM 4.7, DeepSeek V3.2...) |
|
||||
| `oci` | `oci` | OCI Generative AI | API key, enterprise | [link](https://www.oracle.com/artificial-intelligence/generative-ai) | Use your OCI Generative AI API key or IAM bearer token. Base URL can be https://inference.generativeai.<region>.oci.oraclecloud.com/openai/v1/. |
|
||||
| `ofoxai` | `ofoxai` | OfoxAI | API key, aggregator | [link](https://ofox.ai) | The current catalog advertises 10+ free models without a public numeric quota; review upstream provenance, retention and training terms before production use. |
|
||||
| `ollama-cloud` | `ollamacloud` | Ollama Cloud | API key | [link](https://ollama.com/settings/keys) | — |
|
||||
| `openadapter` | `oad` | OpenAdapter | API key | [link](https://openadapter.dev) | Use your OpenAdapter API key in Authorization: Bearer sk-cv-<key>. Fully OpenAI-compatible. API base URL: https://api.openadapter.in/v1. |
|
||||
| `openai` | `openai` | OpenAI | API key | [link](https://platform.openai.com) | — |
|
||||
| `opencode-go` | `opencode-go` | OpenCode Go | API key | [link](https://opencode.ai/go) | — |
|
||||
| `opencode-zen` | `opencode-zen` | OpenCode Zen | API key | [link](https://opencode.ai/zen) | — |
|
||||
| `openference-api` | `ofa` | Openference API | API key | [link](https://openference.com) | Free plan: 3-day trial with open-source models — no credit card required |
|
||||
| `openrouter` | `openrouter` | OpenRouter | API key, aggregator | [link](https://openrouter.ai) | Free models at $0/token with :free suffix - 20 RPM / 200 RPD |
|
||||
| `openvecta` | `openvecta` | OpenVecta | API key | [link](https://openvecta.com) | Free credits on signup for OpenAI-compatible inference across LLMs, embeddings, and reasoning models |
|
||||
| `orcarouter` | `orcarouter` | OrcaRouter | API key | [link](https://www.orcarouter.ai) | — |
|
||||
@@ -241,7 +284,9 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
|
||||
| `pioneer` | `pn` | Pioneer AI | API key | [link](https://pioneer.ai) | $75 free usage credits — no credit card required |
|
||||
| `plamo` | `plamo` | PLaMo | API key | [link](https://plamo.preferredai.jp/api) | — |
|
||||
| `poe` | `poe` | Poe | API key, aggregator | [link](https://creator.poe.com/api-reference) | Bearer API key for the Poe OpenAI-compatible API. |
|
||||
| `pollinations` | `pol` | Pollinations AI | API key, video | [link](https://pollinations.ai) | Free keyless tier: openai, openai-fast, openai-large, qwen-coder, mistral, deepseek, grok, gemini-flash-lite-3.1, perplexity-fast, perplexity-reasoning. Premium models (claude, gemini, midijourney) require a Pollinations API key from enter.pollinations.ai. |
|
||||
| `poixe-ai` | `poixe-ai` | Poixe AI | API key, aggregator | [link](https://poixe.com) | Current public free limits are small and model-group specific: 2 RPM/5 RPD for large-cup models and 20 RPM/50 RPD for small-cup models. |
|
||||
| `pollinations` | `pol` | Pollinations AI | API key, video | [link](https://pollinations.ai) | Anonymous/keyless access to the documented free models is best-effort. Local v3.8.50 verification (2026-07-31) returned 401 via OmniRoute and Cloudflare 1010 on direct upstream probes from the same network. Premium models still require a Pollinations API key from enter.pollinations.ai. |
|
||||
| `poolside` | `poolside` | Poolside | API key | [link](https://poolside.ai) | Laguna S 2.1 and XS 2.1 are free during Preview; no public numeric quota is published. |
|
||||
| `predibase` | `predibase` | Predibase | API key | [link](https://predibase.com) | ⚠️ **DEPRECATED.** serving.app.predibase.com no longer resolves (sweep 2026-06-19); the managed serving API appears discontinued. |
|
||||
| `publicai` | `publicai` | PublicAI | API key | [link](https://publicai.co) | Requires an API key — one-time signup credit, then paid |
|
||||
| `puter` | `pu` | Puter AI | API key | [link](https://puter.com) | Get token at puter.com/dashboard → Copy Auth Token |
|
||||
@@ -250,6 +295,7 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
|
||||
| `qwen-cloud` | `qwc` | Qwen Cloud | API key | [link](https://www.qwencloud.com/) | — |
|
||||
| `qwen-cloud-token-plan` | `qct` | Qwen Cloud Token Plan | API key | [link](https://www.qwencloud.com/pricing/token-plan) | — |
|
||||
| `recraft` | `recraft` | Recraft | API key, image | [link](https://recraft.ai) | — |
|
||||
| `regolo` | `regolo` | Regolo AI | API key | [link](https://regolo.ai) | Get your Regolo API key from regolo.ai, then paste it here as a Bearer token. |
|
||||
| `reka` | `reka` | Reka | API key | [link](https://docs.reka.ai/chat/overview) | Use your Reka API key. OmniRoute supports the OpenAI-compatible base URL https://api.reka.ai/v1 and sends both Authorization and X-Api-Key headers for compatibility. |
|
||||
| `requesty` | `requesty` | Requesty | API key | [link](https://requesty.ai) | Free tier ~200 requests/day - multi-model routing gateway (300+ models) |
|
||||
| `routeway` | `routeway` | Routeway | API key | [link](https://routeway.ai) | Create a free API key at routeway.ai, then paste it here as a Bearer token. |
|
||||
@@ -264,6 +310,7 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
|
||||
| `siliconflow` | `siliconflow` | SiliconFlow | API key | [link](https://cloud.siliconflow.com) | $1 free credits plus permanently free models after identity verification |
|
||||
| `snowflake` | `snowflake` | Snowflake Cortex | API key, enterprise | [link](https://www.snowflake.com) | — |
|
||||
| `sparkdesk` | `sparkdesk` | SparkDesk | API key | [link](https://xinghuo.xfyun.cn) | Get API key at console.xfyun.cn |
|
||||
| `speka` | `speka` | Speka AI | API key, aggregator | [link](https://speka.me) | Free plan: $1 monthly usage, 10 RPM, one API key and access to open models and the playground; no card required. |
|
||||
| `stability-ai` | `stability` | Stability AI | API key, image | [link](https://stability.ai) | — |
|
||||
| `stepfun` | `stepfun` | StepFun | API key | [link](https://stepfun.com) | Get API key at platform.stepfun.com |
|
||||
| `sumopod` | `sumopod` | SumoPod | API key | [link](https://ai.sumopod.com) | Use your SumoPod API key (sk-...) in Authorization: Bearer <key>. Fully OpenAI-compatible. API base URL: https://ai.sumopod.com/v1. |
|
||||
@@ -273,11 +320,13 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
|
||||
| `thebai` | `thebai` | TheB.AI | API key, aggregator | [link](https://theb.ai) | Bearer API key for the TheB.AI OpenAI-compatible gateway. |
|
||||
| `tinyfish` | `tf` | TinyFish Fetch | API key | [link](https://docs.tinyfish.ai/fetch-api) | X-API-Key from agent.tinyfish.ai/api-keys |
|
||||
| `together` | `together` | Together AI | API key, video | [link](https://www.together.ai) | — |
|
||||
| `tokenreply` | `tokenreply` | TokenReply | API key, aggregator | [link](https://www.tokenreply.com) | Free-tagged models have model- and campaign-specific daily limits; no fixed global free quota is published. |
|
||||
| `tokenrouter` | `trk` | TokenRouter | API key | [link](https://tokenrouter.com) | Use your TokenRouter API key in Authorization: Bearer <key>. Fully OpenAI-compatible. API base URL: https://api.tokenrouter.com/v1. |
|
||||
| `topaz` | `topaz` | Topaz | API key, image | [link](https://topazlabs.com) | — |
|
||||
| `typhoon` | `typhoon` | Typhoon | API key | [link](https://docs.opentyphoon.ai) | Free API key with a 5 req/s and 200 req/m rate limit. |
|
||||
| `udio` | `udio` | Udio | API key | [link](https://udio.com) | Paste session cookie from udio.com (Supabase auth) |
|
||||
| `uncloseai` | `unc` | UncloseAI | API key | [link](https://uncloseai.com) | No auth required. API accepts any non-empty string as key for identification. |
|
||||
| `uncloseai` | `unc` | UncloseAI | API key | [link](https://uncloseai.com) | No auth required. API accepts any non-empty string as key for identification. If older built-in models return 404, use Available Models → Import from /models or Auto-Sync; verified live model: solidrust/Hermes-3-Llama-3.1-8B-AWQ. |
|
||||
| `unorouter` | `unorouter` | UnoRouter | API key, aggregator | [link](https://unorouter.ai) | Models with the :free suffix do not debit balance; limit is 1 request/minute per free model per user. |
|
||||
| `upstage` | `upstage` | Upstage | API key | [link](https://www.upstage.ai) | — |
|
||||
| `v0-vercel` | `v0` | v0 (Vercel) | API key | [link](https://v0.dev) | — |
|
||||
| `venice` | `venice` | Venice.ai | API key | [link](https://venice.ai) | — |
|
||||
@@ -295,8 +344,11 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
|
||||
| `xiaomi-mimo` | `mimo` | Xiaomi MiMo | API key | [link](https://mimo.mi.com) | — |
|
||||
| `xiaomi-mimo-token-plan` | `mimotp` | Xiaomi MiMo Token Plan | API key | [link](https://mimo.mi.com) | — |
|
||||
| `yi` | `yi` | Yi (01.AI) | API key | [link](https://01.ai) | Get API key at platform.lingyiwanwu.com |
|
||||
| `yolo-auto` | `yolo-auto` | Yolo-Auto | API key, aggregator | [link](https://yolo-auto.com) | Free API access is request-limited and intended for testing; no numeric daily quota is published and free access is not promised indefinitely. |
|
||||
| `zai` | `zai` | Z.AI | API key | [link](https://open.bigmodel.cn) | — |
|
||||
| `zenmux` | `zm` | ZenMux | API key | [link](https://zenmux.ai) | Use your ZenMux API key in Authorization: Bearer <key>. ZenMux is fully OpenAI-compatible. Base URL: https://zenmux.ai/api/v1. |
|
||||
| `zerolimitai` | `zerolimitai` | ZeroLimitAI | API key, aggregator | [link](https://www.zerolimitai.com) | Temporary free trial is advertised, but official pages conflict between 3 and 7 days; a 100-calls/day claim is not treated as permanent. |
|
||||
| `zylo-api` | `zylo` | Zylo API | API key, aggregator | [link](https://zyloai.net) | Basic plan: 10 RPM, 7,200 requests/day and 200,000 tokens/day; limited to Basic text models. |
|
||||
|
||||
## Local Providers (12)
|
||||
|
||||
@@ -321,7 +373,7 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
|
||||
|----|-------|------|------|---------|-------|
|
||||
| `brave-search` | `brave-search` | Brave Search | Search | [link](https://brave.com/search/api) | Subscription token from Brave Search API dashboard |
|
||||
| `exa-search` | `exa-search` | Exa Search | Search | [link](https://exa.ai) | API key from dashboard.exa.ai |
|
||||
| `firecrawl` | `fc` | Firecrawl | Search | [link](https://firecrawl.dev) | — |
|
||||
| `firecrawl` | `fc` | Firecrawl | Search | [link](https://firecrawl.dev) | API key from firecrawl.dev/app/api-keys (or set your self-hosted Firecrawl base URL) |
|
||||
| `google-pse-search` | `google-pse` | Google Programmable Search | Search | [link](https://developers.google.com/custom-search/v1/overview) | Requires a Google API key and your Programmable Search Engine ID (cx) |
|
||||
| `linkup-search` | `linkup` | Linkup Search | Search | [link](https://docs.linkup.so) | Bearer API key from the Linkup dashboard |
|
||||
| `ollama-search` | `ollama-search` | Ollama Search | Search | [link](https://ollama.com/settings/keys) | Same API key as Ollama Cloud (from ollama.com/settings/keys) |
|
||||
@@ -332,7 +384,7 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
|
||||
| `tavily-search` | `tavily-search` | Tavily Search | Search | [link](https://tavily.com) | API key from app.tavily.com (format: tvly-...) |
|
||||
| `youcom-search` | `youcom-search` | You.com Search | Search | [link](https://you.com/business/api/) | X-API-Key from the You.com platform dashboard |
|
||||
|
||||
## Audio-only Providers (11)
|
||||
## Audio-only Providers (12)
|
||||
|
||||
| ID | Alias | Name | Tags | Website | Notes |
|
||||
|----|-------|------|------|---------|-------|
|
||||
@@ -346,6 +398,7 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
|
||||
| `inworld` | `inworld` | Inworld | Audio | [link](https://inworld.ai) | — |
|
||||
| `playht` | `playht` | PlayHT | Audio | [link](https://play.ht) | — |
|
||||
| `rev-ai` | `revai` | Rev AI | Audio | [link](https://www.rev.ai) | — |
|
||||
| `soniox` | `sx` | Soniox | Audio | [link](https://soniox.com) | — |
|
||||
| `speechmatics` | `sm` | Speechmatics | Audio | [link](https://www.speechmatics.com) | Free tier — 8 hours/month, no credit card required. Batch (async) mode only. |
|
||||
|
||||
## Upstream Proxy Providers (2)
|
||||
@@ -373,7 +426,7 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
|
||||
|
||||
- Catalog: [`src/shared/constants/providers.ts`](../../src/shared/constants/providers.ts)
|
||||
- Registry (per-model details): [`open-sse/config/providerRegistry.ts`](../../open-sse/config/providerRegistry.ts)
|
||||
- Executors: [`open-sse/executors/`](../../open-sse/executors/) (31 files)
|
||||
- Executors: [`open-sse/executors/`](../../open-sse/executors/) (101 implementations)
|
||||
- Translators: [`open-sse/translator/`](../../open-sse/translator/)
|
||||
|
||||
## See Also
|
||||
|
||||
@@ -161,11 +161,11 @@ The detection helper lives in `src/lib/combos/modelNameCollision.ts`.
|
||||
|
||||
## How It Works (Persisted Auto-Combos)
|
||||
|
||||
The Auto-Combo Engine dynamically selects the best provider/model for each request using a **13-factor scoring function** (defined in `open-sse/services/autoCombo/scoring.ts` → `DEFAULT_WEIGHTS`). All weights sum to **1.0**.
|
||||
The Auto-Combo Engine dynamically selects the best provider/model for each request using a **14-factor scoring function** (defined in `open-sse/services/autoCombo/scoring.ts` → `DEFAULT_WEIGHTS`). Weights form a normalized distribution (custom weights are renormalized by `normalizeScoringWeights()`).
|
||||
|
||||
|
||||
|
||||
|
||||
> Source: [diagrams/auto-combo-12factor.mmd](../diagrams/auto-combo-12factor.mmd) (regenerate via `npm run docs:render-diagrams`). Diagram/filename predate the `cacheAffinity` factor added by #8008 and still show 12 factors.
|
||||
> Source: [diagrams/auto-combo-12factor.mmd](../diagrams/auto-combo-12factor.mmd) (regenerate via `npm run docs:render-diagrams`). The filename predates the current factor set; the diagram shows 13 of the 14 factors (missing `sessionAvailability`).
|
||||
|
||||
| Factor | Default Weight | Description |
|
||||
| :-------------------- | :------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
@@ -179,11 +179,12 @@ The Auto-Combo Engine dynamically selects the best provider/model for each reque
|
||||
| `tierAffinity` | 0.05 | Affinity between the candidate's tier and the manifest-recommended tier |
|
||||
| `specificityMatch` | 0.05 | Match between request specificity (manifest hint) and model tier |
|
||||
| `contextAffinity` | 0.05 | Affinity between the request's context-window need and the model's context window |
|
||||
| `sessionAvailability` | 0.05 | OAuth session availability of the candidate connection for this session (`getOAuthSessionAvailability()`; non-OAuth connections score 1.0) |
|
||||
| `connectionDensity` | 0.05 | Spreads load across connections of the same provider (anti-concentration) |
|
||||
| `cacheAffinity` | 0.00 | Rendezvous-hash affinity toward the connection likeliest to already hold this request's prompt-cache prefix (`open-sse/services/combo/promptCacheAffinity.ts`); disabled by default (#8008) |
|
||||
| `resetWindowAffinity` | 0.00 | Bias toward connections whose quota reset window is favorable (disabled by default) |
|
||||
|
||||
**Sum:** `0.20 + 0.15 + 0.15 + 0.12 + 0.08 + 0.05 + 0.05 + 0.05 + 0.05 + 0.05 + 0.05 + 0.00 + 0.00 = 1.0` (validated by `validateWeights()`).
|
||||
**Sum:** `0.20 + 0.15 + 0.15 + 0.12 + 0.08 + 0.05 + 0.05 + 0.05 + 0.05 + 0.05 + 0.05 + 0.05 + 0.00 + 0.00 = 1.05` as literally declared in `DEFAULT_WEIGHTS`; user-configured weights are renormalized into a distribution by `normalizeScoringWeights()` before scoring.
|
||||
|
||||
## Mode Packs
|
||||
|
||||
@@ -654,7 +655,7 @@ Including the bare `auto` (default) plus the 6 `AutoVariant` values declared in
|
||||
|
||||
## How tiers fit Auto-Combo
|
||||
|
||||
The 12-factor scoring function (`open-sse/services/autoCombo/scoring.ts`) treats tier
|
||||
The 14-factor scoring function (`open-sse/services/autoCombo/scoring.ts`) treats tier
|
||||
membership as two signals: `tierPriority` (0.05) and `tierAffinity` (0.05). See the
|
||||
canonical [scoring factor table](#how-it-works-persisted-auto-combos) above for the full
|
||||
`DEFAULT_WEIGHTS` set — the per-pack overrides (ship-fast/cost-saver/quality-first/
|
||||
|
||||
58
llm.txt
58
llm.txt
@@ -1,6 +1,6 @@
|
||||
# OmniRoute
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -8,13 +8,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.47
|
||||
**Current version:** 3.8.50
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
- **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth
|
||||
@@ -22,7 +22,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Background jobs:** Custom token health check scheduler, 24h model auto-sync
|
||||
- **Streaming:** Server-Sent Events (SSE) for real-time proxy responses
|
||||
- **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine
|
||||
- **i18n:** next-intl with 40+ languages
|
||||
- **i18n:** next-intl with 43 languages
|
||||
- **Desktop:** Electron (cross-platform: Windows, macOS, Linux)
|
||||
- **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`)
|
||||
|
||||
@@ -41,7 +41,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -94,7 +94,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── configAudit.ts # Configuration auditing
|
||||
│ │ └── responses.ts # Domain response types
|
||||
│ ├── i18n/ # Internationalization
|
||||
│ │ └── messages/ # 40+ language JSON files
|
||||
│ │ └── messages/ # 43 language JSON files
|
||||
│ ├── lib/ # Core libraries
|
||||
│ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server
|
||||
│ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting)
|
||||
@@ -165,7 +165,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -182,7 +182,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -208,11 +208,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
@@ -224,7 +224,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -263,8 +263,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -275,15 +275,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.47)
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **338 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -312,7 +312,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -342,13 +342,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (99 Tools)
|
||||
### MCP Server (105 Tools)
|
||||
|
||||
99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
@@ -369,7 +369,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 40+ languages for UI (all dashboard pages)
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
@@ -381,7 +381,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
@@ -435,7 +435,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -443,7 +443,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -476,10 +476,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "omniroute",
|
||||
"version": "3.8.50",
|
||||
"description": "Unified AI router with 291 providers, RTK+Caveman compression, auto fallback, MCP/A2A, desktop, PWA, and OpenAI-compatible APIs.",
|
||||
"description": "Unified AI router with 338 providers, RTK+Caveman compression, auto fallback, MCP/A2A, desktop, PWA, and OpenAI-compatible APIs.",
|
||||
"type": "module",
|
||||
"bin": {
|
||||
"omniroute": "bin/omniroute.mjs",
|
||||
|
||||
@@ -18,9 +18,13 @@
|
||||
// Exits 0 on success, 1 on STRICT drift (or any drift with --strict).
|
||||
// Run: node scripts/check/check-docs-counts-sync.mjs
|
||||
//
|
||||
// NOTE: the provider check trusts PROVIDER_REFERENCE.md as the canonical total. If a
|
||||
// provider is added to the code but the reference is not regenerated, this guard will
|
||||
// not catch it — regenerate with `npm run gen:provider-reference` before relying on it.
|
||||
// NOTE: PROVIDER_REFERENCE.md is no longer blindly trusted — a STRICT check compares
|
||||
// the doc's `Total providers` against the live provider modules (the same collections
|
||||
// the generator reads), so a hand-stale doc is a red, not a silently propagated total.
|
||||
// Fix by running `npm run gen:provider-reference`. Additional STRICT coverage added in
|
||||
// the 2026-08-12 hardening: llm.txt + package.json description (providers), migration
|
||||
// count (README/AGENTS/llm.txt), and canonical numbers inside the README SVG diagrams
|
||||
// (providers / MCP tools / routing strategies / free-tier pools).
|
||||
|
||||
import fs from "node:fs";
|
||||
import { spawnSync } from "node:child_process";
|
||||
@@ -76,6 +80,13 @@ export function readProviderTotal() {
|
||||
return parseProviderTotal(fs.readFileSync(abs, "utf8"));
|
||||
}
|
||||
|
||||
// STRICT: number of SQL migration files shipped with the app.
|
||||
export function countMigrations() {
|
||||
const abs = path.join(ROOT, "src", "lib", "db", "migrations");
|
||||
if (!fs.existsSync(abs)) return 0;
|
||||
return fs.readdirSync(abs).filter((f) => f.endsWith(".sql")).length;
|
||||
}
|
||||
|
||||
// STRICT: canonical i18n locale count, read from the shared config.
|
||||
export function countLocales() {
|
||||
const abs = path.join(ROOT, "config", "i18n.json");
|
||||
@@ -148,6 +159,15 @@ function readCodeFacts() {
|
||||
'import {notionTools} from "./open-sse/mcp-server/tools/notionTools.ts";',
|
||||
'import {obsidianTools} from "./open-sse/mcp-server/tools/obsidianTools.ts";',
|
||||
'import {compressionTools} from "./open-sse/mcp-server/tools/compressionTools.ts";',
|
||||
// Live provider total — the SAME collections gen-provider-reference.ts unions, so the
|
||||
// doc-vs-live check below cannot drift from the generator's definition of "provider".
|
||||
'import * as PROV from "./src/shared/constants/providers.ts";',
|
||||
"const provCols=[PROV.FREE_PROVIDERS,PROV.NOAUTH_PROVIDERS,PROV.OAUTH_PROVIDERS,",
|
||||
"PROV.WEB_COOKIE_PROVIDERS,PROV.APIKEY_PROVIDERS,PROV.LOCAL_PROVIDERS,PROV.SEARCH_PROVIDERS,",
|
||||
"PROV.AUDIO_ONLY_PROVIDERS,PROV.UPSTREAM_PROXY_PROVIDERS,PROV.CLOUD_AGENT_PROVIDERS,",
|
||||
"PROV.SYSTEM_PROVIDERS];",
|
||||
"const pids=new Set();",
|
||||
"for(const c of provCols)for(const p of Object.values(c||{}))if(p&&p.id)pids.add(p.id);",
|
||||
"const cols={MCP_TOOLS,memoryTools,skillTools,agentSkillTools,githubSkillTools,poolTools,",
|
||||
"gamificationTools,pluginTools,notionTools,obsidianTools,compressionTools};",
|
||||
"const sc=new Set();",
|
||||
@@ -158,7 +178,7 @@ function readCodeFacts() {
|
||||
'console.log("@@"+JSON.stringify({freeSteady:t.steadyRecurringTokens,',
|
||||
"freeFirst:t.firstMonthRealisticTokens,freePools:t.poolCount,engines:ENGINE_IDS.length,",
|
||||
"cliTotal:cli.length,cliCode:by('code'),cliAgent:by('agent'),",
|
||||
"mcpTools:countUniqueMcpTools(cols),mcpScopes:sc.size}));",
|
||||
"mcpTools:countUniqueMcpTools(cols),mcpScopes:sc.size,providers:pids.size}));",
|
||||
].join("");
|
||||
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "docs-counts-"));
|
||||
try {
|
||||
@@ -252,6 +272,82 @@ export function makeNumberClaimValidator(expected, opts) {
|
||||
};
|
||||
}
|
||||
|
||||
// --- v3.8.50 hardening validators --------------------------------------------
|
||||
// PURE: doc total must equal the live provider-module total (closes the falso-verde
|
||||
// found in the 2026-08-12 audit: the doc sat hand-stale at 291 while the modules
|
||||
// defined 338, and every downstream check inherited the stale total).
|
||||
export function makeProviderReferenceValidator(expected) {
|
||||
return (content) => {
|
||||
const total = parseProviderTotal(content);
|
||||
if (!total) return { ok: false, detail: "no `Total providers: **N**` marker found" };
|
||||
if (total === expected)
|
||||
return { ok: true, detail: `doc total ${total} matches the live provider modules` };
|
||||
return {
|
||||
ok: false,
|
||||
detail:
|
||||
`doc total ${total} is stale — the live provider modules define ${expected} ` +
|
||||
`(run npm run gen:provider-reference)`,
|
||||
};
|
||||
};
|
||||
}
|
||||
|
||||
// PURE: the npm package description must carry the live provider count.
|
||||
export function makePackageDescriptionValidator(expected) {
|
||||
return (content) => {
|
||||
let desc = "";
|
||||
try {
|
||||
desc = String(JSON.parse(content).description || "");
|
||||
} catch {
|
||||
return { ok: false, detail: "package.json could not be parsed" };
|
||||
}
|
||||
if (desc.includes(String(expected)))
|
||||
return { ok: true, detail: `description mentions the live provider count ${expected}` };
|
||||
return {
|
||||
ok: false,
|
||||
detail: `description does not mention the live provider count ${expected}: "${desc}"`,
|
||||
};
|
||||
};
|
||||
}
|
||||
|
||||
// PURE: sweep an SVG's text/aria content for the canonical numbers. Patterns are
|
||||
// deliberately narrow — they anchor on the surrounding words so path coordinates,
|
||||
// width/font-size attributes and small unrelated counts ("15 providers ToS-flagged",
|
||||
// "100+ providers") can never register as claims. Providers require 3+ digits for the
|
||||
// same reason.
|
||||
const SVG_CANONICAL_PATTERNS = [
|
||||
{ key: "providers", what: "providers", pattern: /(\d{3,4}) (?:AI )?providers\b/g },
|
||||
{ key: "mcpTools", what: "MCP tools", pattern: /MCP (?:server with |with |\()(\d+)/g },
|
||||
{ key: "strategies", what: "routing strategies", pattern: /(\d+) routing strategies\b/g },
|
||||
{ key: "pools", what: "free-tier pools", pattern: /(\d+) provider pools\b/g },
|
||||
];
|
||||
|
||||
export function checkSvgCanonicalNumbers(content, expected) {
|
||||
const stale = [];
|
||||
let claims = 0;
|
||||
for (const { key, what, pattern } of SVG_CANONICAL_PATTERNS) {
|
||||
if (expected[key] == null) continue;
|
||||
for (const m of content.matchAll(pattern)) {
|
||||
claims++;
|
||||
const value = Number(m[1]);
|
||||
if (value !== expected[key]) stale.push(`"${m[0]}" (${what} — code has ${expected[key]})`);
|
||||
}
|
||||
}
|
||||
if (!claims) return { ok: true, detail: "no canonical-number claims in this SVG" };
|
||||
if (!stale.length) return { ok: true, detail: `${claims} canonical claim(s) match the code` };
|
||||
return { ok: false, detail: `stale: ${[...new Set(stale)].join(", ")}` };
|
||||
}
|
||||
|
||||
// The README-embedded diagrams that historically rotted because no gate read them
|
||||
// (the alt-text in README.md is checked, the SVG text nodes never were).
|
||||
const SVG_DIAGRAM_FILES = [
|
||||
"docs/diagrams/readme-hero.svg",
|
||||
"docs/diagrams/free-tier-budget.svg",
|
||||
"docs/diagrams/promise-pillars.svg",
|
||||
"docs/diagrams/comparison-table.svg",
|
||||
"docs/diagrams/cli-terminal.svg",
|
||||
"docs/diagrams/tier-cascade.svg",
|
||||
];
|
||||
|
||||
export function buildChecks() {
|
||||
return [
|
||||
{
|
||||
@@ -259,7 +355,26 @@ export function buildChecks() {
|
||||
actual: readProviderTotal(),
|
||||
docKey: "providers",
|
||||
strict: true,
|
||||
files: ["README.md", "AGENTS.md"],
|
||||
files: ["README.md", "AGENTS.md", "llm.txt"],
|
||||
},
|
||||
{
|
||||
label: "Provider count (package.json description)",
|
||||
actual: readProviderTotal(),
|
||||
docKey: "providers",
|
||||
strict: true,
|
||||
files: ["package.json"],
|
||||
validate: makePackageDescriptionValidator(readProviderTotal()),
|
||||
},
|
||||
{
|
||||
label: "DB migrations count",
|
||||
actual: countMigrations(),
|
||||
docKey: "migrations",
|
||||
strict: true,
|
||||
files: ["README.md", "AGENTS.md", "llm.txt"],
|
||||
validate: makeNumberClaimValidator(countMigrations(), {
|
||||
what: "migrations",
|
||||
pattern: /(\d+)\+? migrations?\b/gi,
|
||||
}),
|
||||
},
|
||||
{
|
||||
label: "i18n locales count",
|
||||
@@ -289,6 +404,30 @@ export function buildChecks() {
|
||||
validate: makeNumberClaimValidator(expected, { what, ...opts }),
|
||||
});
|
||||
return [
|
||||
{
|
||||
label: "Provider reference total (doc vs live modules)",
|
||||
actual: f.providers,
|
||||
docKey: "providers (live)",
|
||||
strict: true,
|
||||
files: ["docs/reference/PROVIDER_REFERENCE.md"],
|
||||
validate: makeProviderReferenceValidator(f.providers),
|
||||
},
|
||||
{
|
||||
label: "SVG canonical numbers (live code)",
|
||||
actual:
|
||||
`${f.providers} providers / ${f.mcpTools} MCP tools / ` +
|
||||
`${countRoutingStrategies()} strategies / ${f.freePools} pools`,
|
||||
docKey: "SVG canonical numbers",
|
||||
strict: true,
|
||||
files: SVG_DIAGRAM_FILES,
|
||||
validate: (content) =>
|
||||
checkSvgCanonicalNumbers(content, {
|
||||
providers: f.providers,
|
||||
mcpTools: f.mcpTools,
|
||||
strategies: countRoutingStrategies(),
|
||||
pools: f.freePools,
|
||||
}),
|
||||
},
|
||||
{
|
||||
label: "Free-tier headline (live catalog)",
|
||||
actual: `~${(f.freeSteady / 1e9).toFixed(2)}B steady / ${f.freePools} pools`,
|
||||
|
||||
@@ -7,6 +7,7 @@ import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import {
|
||||
FREE_PROVIDERS,
|
||||
NOAUTH_PROVIDERS,
|
||||
OAUTH_PROVIDERS,
|
||||
WEB_COOKIE_PROVIDERS,
|
||||
APIKEY_PROVIDERS,
|
||||
@@ -136,6 +137,7 @@ function buildHeader(total: number): string {
|
||||
"## Categories",
|
||||
"",
|
||||
"- **Free** — free tier with API key (configured via dashboard)",
|
||||
"- **No-auth** — public endpoints that require no key or sign-in at all",
|
||||
"- **OAuth** — sign-in flow handled by OmniRoute, no API key needed",
|
||||
"- **Web cookie** — wraps the provider's web app via cookie auth",
|
||||
"- **API key** — paid provider configured via API key (free credits may apply)",
|
||||
@@ -159,8 +161,19 @@ function buildHeader(total: number): string {
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
function countExecutorImpls(): number {
|
||||
const dir = path.join(ROOT, "open-sse", "executors");
|
||||
const nonImpl = new Set(["index.ts", "index.mts", "types.ts", "base.ts", "constants.ts"]);
|
||||
return fs
|
||||
.readdirSync(dir)
|
||||
.filter(
|
||||
(f) => f.endsWith(".ts") && !f.endsWith(".test.ts") && !f.startsWith("__") && !nonImpl.has(f)
|
||||
).length;
|
||||
}
|
||||
|
||||
function main() {
|
||||
const free = asRecords(FREE_PROVIDERS);
|
||||
const noauth = asRecords(NOAUTH_PROVIDERS as Record<string, ProviderRecord>);
|
||||
const oauth = asRecords(OAUTH_PROVIDERS);
|
||||
const webCookie = asRecords(WEB_COOKIE_PROVIDERS);
|
||||
const apiKey = asRecords(APIKEY_PROVIDERS);
|
||||
@@ -173,6 +186,7 @@ function main() {
|
||||
|
||||
const allIds = new Set<string>([
|
||||
...free.map((p) => p.id),
|
||||
...noauth.map((p) => p.id),
|
||||
...oauth.map((p) => p.id),
|
||||
...webCookie.map((p) => p.id),
|
||||
...apiKey.map((p) => p.id),
|
||||
@@ -186,6 +200,7 @@ function main() {
|
||||
|
||||
const sections = [
|
||||
buildSection("Free Tier (OAuth-first or no-key)", free, "Free"),
|
||||
buildSection("No-auth Providers (no key required)", noauth, "No-auth"),
|
||||
buildSection("OAuth Providers", oauth, "OAuth"),
|
||||
buildSection("Web Cookie Providers", webCookie, "Web cookie"),
|
||||
buildSection("API Key Providers (paid / paid-with-free-credits)", apiKey, "API key"),
|
||||
@@ -202,7 +217,7 @@ function main() {
|
||||
"",
|
||||
"- Catalog: [`src/shared/constants/providers.ts`](../../src/shared/constants/providers.ts)",
|
||||
"- Registry (per-model details): [`open-sse/config/providerRegistry.ts`](../../open-sse/config/providerRegistry.ts)",
|
||||
"- Executors: [`open-sse/executors/`](../../open-sse/executors/) (31 files)",
|
||||
`- Executors: [\`open-sse/executors/\`](../../open-sse/executors/) (${countExecutorImpls()} implementations)`,
|
||||
"- Translators: [`open-sse/translator/`](../../open-sse/translator/)",
|
||||
"",
|
||||
"## See Also",
|
||||
@@ -218,9 +233,10 @@ function main() {
|
||||
console.log(`✓ Wrote ${OUT_FILE}`);
|
||||
console.log(` Providers: ${allIds.size} unique IDs`);
|
||||
console.log(
|
||||
` Sections: free=${free.length}, oauth=${oauth.length}, web=${webCookie.length}, ` +
|
||||
`apikey=${apiKey.length}, local=${local.length}, search=${search.length}, ` +
|
||||
`audio=${audio.length}, proxy=${upstreamProxy.length}, cloud=${cloudAgent.length}, system=${system.length}`
|
||||
` Sections: free=${free.length}, noauth=${noauth.length}, oauth=${oauth.length}, ` +
|
||||
`web=${webCookie.length}, apikey=${apiKey.length}, local=${local.length}, ` +
|
||||
`search=${search.length}, audio=${audio.length}, proxy=${upstreamProxy.length}, ` +
|
||||
`cloud=${cloudAgent.length}, system=${system.length}`
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -196,3 +196,87 @@ test("compression-engines and CLI-tools gates catch their v3.8.49 drift", () =>
|
||||
assert.equal(cli("all 33 tools (25 CLI Code's)").ok, true);
|
||||
assert.equal(cli("all 26 tools (25 CLI Code's)").ok, false);
|
||||
});
|
||||
|
||||
// --- v3.8.50 hardening: live provider source, llm.txt/package.json, migrations, SVGs --
|
||||
// Regression guards for the 2026-08-12 audit: PROVIDER_REFERENCE.md was hand-stale at
|
||||
// 291 while the live provider modules defined 338, and the gate trusted the doc — so
|
||||
// README/AGENTS validated against a stale total and the gate stayed falsely green.
|
||||
import {
|
||||
countMigrations,
|
||||
makeProviderReferenceValidator,
|
||||
makePackageDescriptionValidator,
|
||||
checkSvgCanonicalNumbers,
|
||||
} from "../../scripts/check/check-docs-counts-sync.mjs";
|
||||
|
||||
const countMigs = countMigrations as () => number;
|
||||
const makeRefValidator = makeProviderReferenceValidator as (
|
||||
expected: number
|
||||
) => (content: string) => { ok: boolean; detail: string };
|
||||
const makePkgValidator = makePackageDescriptionValidator as (
|
||||
expected: number
|
||||
) => (content: string) => { ok: boolean; detail: string };
|
||||
const checkSvg = checkSvgCanonicalNumbers as (
|
||||
content: string,
|
||||
expected: { providers?: number; mcpTools?: number; strategies?: number; pools?: number }
|
||||
) => { ok: boolean; detail: string };
|
||||
|
||||
test("countMigrations reads a real, positive migration count", () => {
|
||||
assert.ok(countMigs() > 100, "migrations dir should hold > 100 .sql files");
|
||||
});
|
||||
|
||||
test("provider-reference validator accepts the live total and rejects a stale doc", () => {
|
||||
const v = makeRefValidator(338);
|
||||
assert.equal(v("Total providers: **338**. See category breakdown below.").ok, true);
|
||||
const stale = v("Total providers: **291**. See category breakdown below.");
|
||||
assert.equal(stale.ok, false, "a hand-stale doc total must be a red, not a silent pass");
|
||||
assert.match(stale.detail, /gen:provider-reference/);
|
||||
assert.equal(v("# Provider Reference\n\nNo total marker.").ok, false);
|
||||
});
|
||||
|
||||
test("package.json description validator catches a stale provider count", () => {
|
||||
const v = makePkgValidator(338);
|
||||
assert.equal(v(JSON.stringify({ description: "Unified AI router with 338 providers" })).ok, true);
|
||||
assert.equal(
|
||||
v(JSON.stringify({ description: "Unified AI router with 291 providers" })).ok,
|
||||
false
|
||||
);
|
||||
assert.equal(v("not json at all {").ok, false);
|
||||
});
|
||||
|
||||
test("migrations claim validator accepts the real count and rejects stale styles", () => {
|
||||
const v = makeValidator(144, { what: "migrations", pattern: /(\d+)\+? migrations?\b/gi });
|
||||
assert.equal(v("SQLite domain modules (144 migrations)").ok, true);
|
||||
assert.equal(v("local, zero-config, 110+ migrations").ok, false);
|
||||
assert.equal(v("(130 migrations)").ok, false);
|
||||
});
|
||||
|
||||
const SVG_EXPECTED = { providers: 338, mcpTools: 105, strategies: 19, pools: 42 };
|
||||
|
||||
test("SVG gate accepts canonical numbers in text and aria-label claims", () => {
|
||||
const good =
|
||||
'aria-label="338 AI providers, 19 routing strategies, MCP with 105 tools, ' +
|
||||
'42 provider pools" <text>338 providers</text><text>MCP (105</text>';
|
||||
assert.equal(checkSvg(good, SVG_EXPECTED).ok, true);
|
||||
});
|
||||
|
||||
test("SVG gate flags each stale canonical number the audit found", () => {
|
||||
for (const stale of [
|
||||
"<text>290 providers</text>",
|
||||
'aria-label="MCP server with 104 tools"',
|
||||
"<text>MCP (104</text>",
|
||||
"<text>18 routing strategies</text>",
|
||||
'aria-label="43 provider pools and 460+ models"',
|
||||
]) {
|
||||
const r = checkSvg(stale, SVG_EXPECTED);
|
||||
assert.equal(r.ok, false, `expected stale claim to be flagged: ${stale}`);
|
||||
}
|
||||
});
|
||||
|
||||
test("SVG gate ignores coordinates, sizes and unrelated small counts", () => {
|
||||
const noise =
|
||||
'<path d="M 30,310 C 136,290 176,240 296,206"/><rect width="104" height="24"/>' +
|
||||
'<animate values="250;290;250"/><text font-size="104">15 providers ToS-flagged</text>' +
|
||||
"<text>100+ providers</text><text>90+ free</text>";
|
||||
const r = checkSvg(noise, SVG_EXPECTED);
|
||||
assert.equal(r.ok, true, `coordinates/attrs must never register claims: ${r.detail}`);
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user