mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-18 05:02:15 +03:00
Merge remote-tracking branch 'origin/release/v3.8.47' into HEAD
This commit is contained in:
Diego Rodrigues de Sa e Souza
1
changelog.d/features/6776-discover-live-codex-models.md
Normal file
1
changelog.d/features/6776-discover-live-codex-models.md
Normal file
@@ -0,0 +1 @@
|
||||
- **feat(codex):** Codex provider model discovery now fetches the live catalog from `chatgpt.com/backend-api/codex/models` using Codex-shaped headers, falling back to a GitHub-hosted model manifest and then to the local static catalog when the live/GitHub sources are unavailable or return an unexpected shape — new `src/app/api/providers/[id]/models/discovery/codex.ts` (normalization, version-gating, merge/enrich against the local catalog) covered by `tests/unit/provider-models-discovery-split.test.ts` and `tests/unit/provider-models-route-codex.test.ts` (#6776 — thanks @JxnLexn).
|
||||
1
changelog.d/fixes/6723-i18n-pt-br-backfill.md
Normal file
1
changelog.d/fixes/6723-i18n-pt-br-backfill.md
Normal file
@@ -0,0 +1 @@
|
||||
- **fix(i18n):** pt-BR was missing 194 UI keys present in `en.json` — a real, silent data-sync gap, not covered by any duplicate/mislabeled #6694 (that issue's 9 `providers.*` keys are disjoint, present-but-untranslated sentinels caused by a separate `providerText()` fallback bug) ([#6695](https://github.com/diegosouzapw/OmniRoute/issues/6695)) — `scripts/i18n/sync-ui-keys.mjs` (which mirrors newly-added `en.json` keys into every locale) wasn't re-run after recent `en.json` additions, and the CI `i18n:check-ui-coverage` gate only fails a locale below an 80% threshold, so pt-BR stayed green at 93.8% coverage despite the gap. Backfilled all 194 missing keys into `src/i18n/messages/pt-BR.json` (translated to Brazilian Portuguese, no leftover `__MISSING__` markers) via `npm run i18n:sync-ui -- --locale=pt-BR` + manual translation. Regression guard: `tests/unit/i18n-pt-br.test.ts` (new case asserting full `en.json`→`pt-BR.json` key parity, so a future drift fails a fast unit test instead of silently degrading the coverage percentage).
|
||||
@@ -0,0 +1 @@
|
||||
- **docs:** refresh `llm.txt` to the current project state (248 providers, 94 MCP tools / 30 scopes, 18 routing strategies, 12-factor Auto-Combo scoring, v3.8.47) and sync its 42 i18n mirrors; move the implemented design-system plan from the repo root to `docs/architecture/DESIGN_SYSTEM.md` rewritten as a reference doc.
|
||||
@@ -1774,7 +1774,7 @@
|
||||
},
|
||||
"tests/unit/provider-models-route.test.ts": {
|
||||
"@typescript-eslint/no-explicit-any": {
|
||||
"count": 57
|
||||
"count": 55
|
||||
}
|
||||
},
|
||||
"tests/unit/provider-models-token-limits.test.ts": {
|
||||
|
||||
@@ -1,7 +1,18 @@
|
||||
---
|
||||
title: "Design System & Visual Identity"
|
||||
lastUpdated: 2026-07-11
|
||||
---
|
||||
|
||||
# OmniRoute — Design System & Visual Identity
|
||||
|
||||
> **Status:** standardization plan. **Phases 1–3 are implemented in this PR** (grid, primitives, status-color centralization, mono token, and the DataTable token migration). The DataTable migration is **faithful** — dark stays byte-identical (the new `--table-*` dark values equal the old hardcoded rgba); light is fixed (it was buggy always-dark via dead `var()` fallbacks). **⚠️ Wants a visual pass before merge** (light-theme tables + the secondary-text shift `#888`→`--color-text-muted`). **Phase 4 is now largely done too** (C6 focus-ring → accent, C7 Checkbox/Textarea primitives, C9 `cn()`→tailwind-merge); only the selective C8 hex-sweep remains. Note several remaining "hardcoded" hex are _intentional_ (always-dark console terminal, ReactFlow SVG strokes) and must NOT be swept. **Phase 5 (D4 + D8): the grid now reaches every standalone screen** (login/auth/error/legal/status/onboarding — their opaque `bg-bg` full-screen wrappers were hiding it) **and the dashboard content shell is fluid up to 4K** (`max-w-7xl` → `max-w-[3840px]`) so it follows the viewport on large monitors instead of centering with wide side gutters. **Phase 6 (D9): data tables are now opaque surfaces** so the grid no longer bleeds through their rows — card-less tables paint `bg-surface`, and the two log tables' semi-transparent `bg-black/5` tint (which tailwind-merge let win over the Card's `bg-surface`) is removed. The grid size itself is already correct (32px, identical to the site); a "bigger" grid on a running instance is a stale build, not code.
|
||||
> **Date:** 2026-06-16 · **Scope:** unify the OmniRoute dashboard (`src/`) with the marketing site (`_mono_repo/omnirouteSite/`) into **one visual identity** — same graph-paper grid background, same color tokens, standardized components.
|
||||
> **Status:** reference — the standardization described here is **implemented** (phases 1–6: grid wallpaper, primitives, status-color centralization, mono token, DataTable token migration, focus-ring → accent, Checkbox/Textarea primitives, `cn()` → tailwind-merge, grid on every standalone screen, fluid 4K content shell, opaque data-table surfaces). This document is the canonical description of the dashboard's design tokens, components, and conventions; the phase framing below is kept as the rationale for each decision.
|
||||
> **Scope:** the OmniRoute dashboard (`src/`) and the marketing site (`_mono_repo/omnirouteSite/`) share **one visual identity** — same graph-paper grid background (32px), same color tokens, standardized components.
|
||||
>
|
||||
> Practical notes for maintainers:
|
||||
>
|
||||
> - Several remaining hardcoded hex values are **intentional** (always-dark console terminal, ReactFlow SVG strokes) and must **NOT** be swept into tokens.
|
||||
> - A "bigger" grid on a running instance is a stale build, not code — the grid size is 32px, identical to the site.
|
||||
> - Dark-theme `--table-*` values are byte-identical to the pre-migration hardcoded rgba; light theme was fixed (it was buggy always-dark via dead `var()` fallbacks).
|
||||
|
||||
---
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
115
llm.txt
115
llm.txt
@@ -1,6 +1,6 @@
|
||||
# OmniRoute
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -8,12 +8,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.8
|
||||
**Current version:** 3.8.47
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 5.9
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
- **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons
|
||||
@@ -41,7 +41,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── auto-combo/ # Auto-combo engine dashboard
|
||||
│ │ │ ├── cache/ # Cache dashboard (semantic cache stats)
|
||||
│ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.)
|
||||
│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates)
|
||||
│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates)
|
||||
│ │ │ ├── costs/ # Cost tracking per provider/model
|
||||
│ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs
|
||||
│ │ │ ├── health/ # System health (uptime, circuit breakers, latency)
|
||||
@@ -102,7 +102,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (95+ modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (99 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -124,7 +124,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 110+ versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 117 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -165,7 +165,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -182,7 +182,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (31 executors)
|
||||
│ ├── executors/ # Provider-specific request executors (78 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -208,15 +208,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── moderations.ts # Content moderation
|
||||
│ │ ├── rerank.ts # Reranking API
|
||||
│ │ └── search.ts # Web search API
|
||||
│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP)
|
||||
│ │ ├── server.ts # MCP server core (tool registration, scope enforcement)
|
||||
│ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools)
|
||||
│ │ ├── schemas/ # Zod input schemas (tools, audit, a2a)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes)
|
||||
│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes)
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 36+ service modules
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -224,7 +224,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -253,24 +253,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 122 unit test files
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
│ ├── translator/ # Translator-specific tests
|
||||
│ └── load/ # Load tests
|
||||
├── docs/ # Documentation
|
||||
│ ├── i18n/ # 30-language translated docs
|
||||
│ ├── ARCHITECTURE.md # Full architecture documentation
|
||||
│ ├── API_REFERENCE.md # API reference
|
||||
│ ├── USER_GUIDE.md # User guide
|
||||
│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview
|
||||
│ ├── CLI-TOOLS.md # CLI tools integration guide
|
||||
│ ├── A2A-SERVER.md # A2A agent protocol documentation
|
||||
│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring)
|
||||
│ ├── MCP-SERVER.md # MCP server (29 tools)
|
||||
│ ├── TROUBLESHOOTING.md # Troubleshooting guide
|
||||
│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide
|
||||
│ ├── i18n/ # 43-language translated docs
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
│ ├── openapi.yaml # OpenAPI specification
|
||||
│ └── screenshots/ # Dashboard screenshots
|
||||
├── bin/ # CLI entry points (omniroute, reset-password)
|
||||
@@ -278,15 +275,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.8.8)
|
||||
## Key Features (v3.8.47)
|
||||
|
||||
### Core Proxy
|
||||
- **177 AI providers** with automatic format translation
|
||||
- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible)
|
||||
- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8)
|
||||
- **248 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
@@ -299,7 +296,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%)
|
||||
- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches
|
||||
|
||||
### Security
|
||||
- **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots.
|
||||
@@ -315,7 +312,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
- **Analytics** — Token consumption, cost, heatmaps, distributions
|
||||
- **Health** — Uptime, memory, latency percentiles, circuit breakers
|
||||
@@ -345,18 +342,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (37 Tools)
|
||||
| Category | Tools |
|
||||
|------------|-------|
|
||||
| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) |
|
||||
| Memory (3) | `memory_search`, `memory_add`, `memory_clear` |
|
||||
| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` |
|
||||
### MCP Server (94 Tools)
|
||||
|
||||
**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`.
|
||||
94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**,
|
||||
**gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
@@ -381,17 +381,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation.
|
||||
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity.
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -435,15 +435,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches.
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes.
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
@@ -474,21 +474,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope.
|
||||
- **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope.
|
||||
|
||||
## v3.8.0 Highlights
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement
|
||||
- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection
|
||||
- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions)
|
||||
- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest
|
||||
- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report
|
||||
- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes
|
||||
- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82%
|
||||
- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams
|
||||
- **Compliance + Evals + Webhooks** documentation introduced
|
||||
- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration
|
||||
- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management
|
||||
- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing
|
||||
- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
## Links
|
||||
|
||||
|
||||
@@ -297,6 +297,7 @@ import {
|
||||
import { resolveBackgroundTaskRedirect } from "./chatCore/backgroundRedirect.ts";
|
||||
import type { CompressionConfig, CompressionPipelineStep } from "../services/compression/types.ts";
|
||||
import { prepareWebSearchFallbackBody } from "../services/webSearchFallback.ts";
|
||||
import { resolveInterceptSearch } from "@/lib/db/interceptionRules";
|
||||
import {
|
||||
resolveExplicitStreamAlias,
|
||||
resolveStreamFlag,
|
||||
@@ -732,12 +733,17 @@ export async function handleChatCore({
|
||||
// Initialize rate limit settings from persisted DB (once, lazy)
|
||||
await initializeRateLimits();
|
||||
|
||||
// #3384: per-model interception rule (src/lib/db/interceptionRules.ts) overrides the
|
||||
// native-bypass defaults below when the operator explicitly configured it for this
|
||||
// provider/model pair; undefined falls through to the existing bypass logic.
|
||||
const interceptSearchOverride = resolveInterceptSearch(provider, effectiveModel);
|
||||
const { body: bodyWithWebSearchFallback, fallback: webSearchFallbackPlan } =
|
||||
prepareWebSearchFallbackBody(body as Record<string, unknown>, {
|
||||
provider,
|
||||
sourceFormat,
|
||||
targetFormat,
|
||||
nativeCodexPassthrough,
|
||||
interceptSearchOverride,
|
||||
});
|
||||
if (webSearchFallbackPlan.enabled) {
|
||||
body = bodyWithWebSearchFallback as typeof body;
|
||||
|
||||
@@ -142,12 +142,20 @@ export function supportsNativeWebSearchFallbackBypass({
|
||||
sourceFormat,
|
||||
targetFormat,
|
||||
nativeCodexPassthrough,
|
||||
interceptSearchOverride,
|
||||
}: {
|
||||
provider?: string | null;
|
||||
sourceFormat?: string | null;
|
||||
targetFormat: string | null | undefined;
|
||||
nativeCodexPassthrough: boolean;
|
||||
// Per-model rule (#3384) — resolveInterceptSearch() in src/lib/db/interceptionRules.ts.
|
||||
// true = force interception (never bypass); false = force native bypass; undefined =
|
||||
// fall through to the native-bypass defaults below.
|
||||
interceptSearchOverride?: boolean;
|
||||
}): boolean {
|
||||
if (typeof interceptSearchOverride === "boolean") {
|
||||
return !interceptSearchOverride;
|
||||
}
|
||||
// Native Codex (OpenAI Responses) passthrough: the upstream runs web search itself.
|
||||
if (nativeCodexPassthrough) return true;
|
||||
// Gemini target: the Gemini translator maps built-in web search to googleSearch natively.
|
||||
@@ -171,6 +179,7 @@ export function prepareWebSearchFallbackBody<T extends JsonRecord>(
|
||||
sourceFormat?: string | null;
|
||||
targetFormat?: string | null;
|
||||
nativeCodexPassthrough: boolean;
|
||||
interceptSearchOverride?: boolean;
|
||||
}
|
||||
): { body: T; fallback: WebSearchFallbackPlan } {
|
||||
const tools = Array.isArray(body.tools) ? body.tools : null;
|
||||
|
||||
@@ -83,10 +83,7 @@ export default function ProviderDetailPageClient() {
|
||||
isAnthropicCompatibleProvider(providerId) && !isClaudeCodeCompatibleProvider(providerId);
|
||||
const isCompatible = isOpenAICompatible || isAnthropicCompatible || isCcCompatible;
|
||||
const isAnthropicProtocolCompatible = isAnthropicCompatible || isCcCompatible;
|
||||
// #5420: hide model listing for tool-only providers (web search / web fetch),
|
||||
// not just `-search`-suffixed ids. Declared serviceKinds come from the static
|
||||
// provider catalog (e.g. firecrawl → ["webFetch"]); compatible providers resolve
|
||||
// to null here and fall through to the empty-kinds check (model listing stays on).
|
||||
// #5420: hide model listing for tool-only providers, not just `-search` ids.
|
||||
const declaredServiceKinds = (
|
||||
resolveDashboardProviderInfo(providerId) as { serviceKinds?: readonly string[] } | null
|
||||
)?.serviceKinds;
|
||||
@@ -224,8 +221,6 @@ export default function ProviderDetailPageClient() {
|
||||
// Prefer synced API-discovered models when available, then merge built-ins
|
||||
// and user-managed custom models without duplicating IDs.
|
||||
const models = useMemo(() => {
|
||||
// Universal: merge built-in registry models with API-synced models and
|
||||
// user-managed custom models for ALL providers (was previously Gemini-only).
|
||||
// Synced models keep their full property spread so provider-specific fields
|
||||
// (e.g. Gemini's `supportedGenerationMethods`) survive into the table.
|
||||
const builtInModels = registryModels.map((model) => ({
|
||||
|
||||
388
src/app/api/providers/[id]/models/discovery/codex.ts
Normal file
388
src/app/api/providers/[id]/models/discovery/codex.ts
Normal file
@@ -0,0 +1,388 @@
|
||||
import {
|
||||
getCodexClientVersion,
|
||||
getCodexDefaultHeaders,
|
||||
} from "@omniroute/open-sse/config/codexClient.ts";
|
||||
|
||||
export const CODEX_MODELS_URL = "https://chatgpt.com/backend-api/codex/models";
|
||||
export const CODEX_GITHUB_MODELS_URL =
|
||||
"https://raw.githubusercontent.com/openai/codex/refs/heads/main/codex-rs/models-manager/models.json";
|
||||
export const CODEX_GITHUB_CATALOG_CACHE_TTL_MS = 5 * 60 * 1000;
|
||||
|
||||
type JsonRecord = Record<string, unknown>;
|
||||
|
||||
export type CodexDiscoveryModel = {
|
||||
id: string;
|
||||
name: string;
|
||||
owned_by: "codex";
|
||||
apiFormat: "responses";
|
||||
supportedEndpoints: ["responses"];
|
||||
inputTokenLimit?: number;
|
||||
outputTokenLimit?: number;
|
||||
description?: string;
|
||||
supportsThinking?: boolean;
|
||||
supportsVision?: boolean;
|
||||
};
|
||||
|
||||
export type CodexModelsFetch = (
|
||||
input: string,
|
||||
init: {
|
||||
method: "GET";
|
||||
headers: Record<string, string>;
|
||||
}
|
||||
) => Promise<Response>;
|
||||
|
||||
type CodexGithubCatalogCache = {
|
||||
models: CodexDiscoveryModel[];
|
||||
etag?: string;
|
||||
expiresAt: number;
|
||||
};
|
||||
|
||||
let codexGithubCatalogCache: CodexGithubCatalogCache | null = null;
|
||||
|
||||
function asRecord(value: unknown): JsonRecord {
|
||||
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
|
||||
}
|
||||
|
||||
function toNonEmptyString(value: unknown): string | null {
|
||||
return typeof value === "string" && value.trim().length > 0 ? value.trim() : null;
|
||||
}
|
||||
|
||||
function firstPositiveNumber(...candidates: unknown[]): number | undefined {
|
||||
for (const candidate of candidates) {
|
||||
if (typeof candidate === "number" && Number.isFinite(candidate) && candidate > 0) {
|
||||
return candidate;
|
||||
}
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
function parseVersionParts(version: string): number[] | null {
|
||||
const parts = version
|
||||
.trim()
|
||||
.split(".")
|
||||
.map((part) => Number(part));
|
||||
return parts.length > 0 && parts.every((part) => Number.isInteger(part) && part >= 0)
|
||||
? parts
|
||||
: null;
|
||||
}
|
||||
|
||||
function compareVersions(left: string, right: string): number {
|
||||
const leftParts = parseVersionParts(left);
|
||||
const rightParts = parseVersionParts(right);
|
||||
if (!leftParts || !rightParts) return 0;
|
||||
|
||||
const length = Math.max(leftParts.length, rightParts.length);
|
||||
for (let index = 0; index < length; index += 1) {
|
||||
const a = leftParts[index] || 0;
|
||||
const b = rightParts[index] || 0;
|
||||
if (a !== b) return a - b;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
export function buildCodexModelsUrl(clientVersion = getCodexClientVersion()): string {
|
||||
const url = new URL(CODEX_MODELS_URL);
|
||||
url.searchParams.set("client_version", clientVersion);
|
||||
return url.toString();
|
||||
}
|
||||
|
||||
function getCodexModelItems(payload: unknown): unknown[] {
|
||||
const record = asRecord(payload);
|
||||
if (Array.isArray(record.models)) return record.models;
|
||||
if (Array.isArray(record.data)) return record.data;
|
||||
if (Array.isArray(payload)) return payload;
|
||||
|
||||
const objectItems = Object.entries(record)
|
||||
.filter(([, value]) => value && typeof value === "object" && !Array.isArray(value))
|
||||
.map(([key, value]) => ({ id: key, ...asRecord(value) }));
|
||||
return objectItems.length > 0 ? objectItems : [];
|
||||
}
|
||||
|
||||
function shouldImportCodexModel(record: JsonRecord): boolean {
|
||||
if (toNonEmptyString(record.visibility)?.toLowerCase() === "hide") return false;
|
||||
if (record.supported_in_api === false || record.supportedInApi === false) return false;
|
||||
|
||||
const minimalClientVersion =
|
||||
toNonEmptyString(record.minimal_client_version) ||
|
||||
toNonEmptyString(record.minimalClientVersion);
|
||||
if (minimalClientVersion && compareVersions(minimalClientVersion, getCodexClientVersion()) > 0) {
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
function getCodexModelId(record: JsonRecord): string | null {
|
||||
return (
|
||||
toNonEmptyString(record.slug) || toNonEmptyString(record.id) || toNonEmptyString(record.model)
|
||||
);
|
||||
}
|
||||
|
||||
function getCodexModelName(record: JsonRecord, id: string): string {
|
||||
return (
|
||||
toNonEmptyString(record.display_name) ||
|
||||
toNonEmptyString(record.displayName) ||
|
||||
toNonEmptyString(record.name) ||
|
||||
toNonEmptyString(record.title) ||
|
||||
id
|
||||
);
|
||||
}
|
||||
|
||||
function recordSupportsThinking(record: JsonRecord): boolean {
|
||||
return (
|
||||
Array.isArray(record.supported_reasoning_levels) && record.supported_reasoning_levels.length > 0
|
||||
);
|
||||
}
|
||||
|
||||
function isImageModality(modality: unknown): boolean {
|
||||
return toNonEmptyString(modality)?.toLowerCase() === "image";
|
||||
}
|
||||
|
||||
function recordSupportsVision(record: JsonRecord): boolean {
|
||||
return Array.isArray(record.input_modalities) && record.input_modalities.some(isImageModality);
|
||||
}
|
||||
|
||||
function buildCodexDiscoveryModel(record: JsonRecord): CodexDiscoveryModel | null {
|
||||
if (!shouldImportCodexModel(record)) return null;
|
||||
|
||||
const id = getCodexModelId(record);
|
||||
if (!id) return null;
|
||||
|
||||
const topProvider = asRecord(record.top_provider);
|
||||
const limits = asRecord(record.limits);
|
||||
const model: CodexDiscoveryModel = {
|
||||
id,
|
||||
name: getCodexModelName(record, id),
|
||||
owned_by: "codex",
|
||||
apiFormat: "responses",
|
||||
supportedEndpoints: ["responses"],
|
||||
};
|
||||
const inputTokenLimit = firstPositiveNumber(
|
||||
record.inputTokenLimit,
|
||||
record.maxInputTokens,
|
||||
record.max_input_tokens,
|
||||
record.contextLength,
|
||||
record.context_length,
|
||||
record.context_window,
|
||||
record.max_context_window,
|
||||
topProvider.context_length,
|
||||
limits.input_tokens,
|
||||
limits.inputTokenLimit,
|
||||
limits.max_input_tokens
|
||||
);
|
||||
const outputTokenLimit = firstPositiveNumber(
|
||||
record.outputTokenLimit,
|
||||
record.maxOutputTokens,
|
||||
record.max_output_tokens,
|
||||
topProvider.max_completion_tokens,
|
||||
limits.output_tokens,
|
||||
limits.outputTokenLimit,
|
||||
limits.max_output_tokens
|
||||
);
|
||||
const description = toNonEmptyString(record.description);
|
||||
|
||||
if (typeof inputTokenLimit === "number") model.inputTokenLimit = inputTokenLimit;
|
||||
if (typeof outputTokenLimit === "number") model.outputTokenLimit = outputTokenLimit;
|
||||
if (description) model.description = description;
|
||||
if (recordSupportsThinking(record)) model.supportsThinking = true;
|
||||
if (recordSupportsVision(record)) model.supportsVision = true;
|
||||
|
||||
return model;
|
||||
}
|
||||
|
||||
export function normalizeCodexModelsResponse(payload: unknown): CodexDiscoveryModel[] {
|
||||
const deduped = new Map<string, CodexDiscoveryModel>();
|
||||
|
||||
for (const item of getCodexModelItems(payload)) {
|
||||
const model = buildCodexDiscoveryModel(asRecord(item));
|
||||
if (model) deduped.set(model.id, model);
|
||||
}
|
||||
|
||||
return Array.from(deduped.values());
|
||||
}
|
||||
|
||||
export function normalizeCodexGithubCatalogResponse(payload: unknown): CodexDiscoveryModel[] {
|
||||
return normalizeCodexModelsResponse(payload);
|
||||
}
|
||||
|
||||
export function clearCodexGithubCatalogCacheForTests(): void {
|
||||
codexGithubCatalogCache = null;
|
||||
}
|
||||
|
||||
function getFreshCodexGithubCatalogCache(
|
||||
now: number,
|
||||
cacheTtlMs: number
|
||||
): CodexDiscoveryModel[] | null {
|
||||
if (cacheTtlMs > 0 && codexGithubCatalogCache?.expiresAt > now) {
|
||||
return codexGithubCatalogCache.models;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function buildCodexGithubCatalogHeaders(): Record<string, string> {
|
||||
const headers: Record<string, string> = {
|
||||
Accept: "application/json",
|
||||
"Content-Type": "application/json",
|
||||
};
|
||||
if (codexGithubCatalogCache?.etag) {
|
||||
headers["If-None-Match"] = codexGithubCatalogCache.etag;
|
||||
}
|
||||
return headers;
|
||||
}
|
||||
|
||||
function getNotModifiedCodexGithubCatalog(
|
||||
response: Response,
|
||||
now: number,
|
||||
cacheTtlMs: number
|
||||
): CodexDiscoveryModel[] | null {
|
||||
if (response.status !== 304 || !codexGithubCatalogCache) return null;
|
||||
|
||||
codexGithubCatalogCache = {
|
||||
...codexGithubCatalogCache,
|
||||
expiresAt: now + cacheTtlMs,
|
||||
};
|
||||
return codexGithubCatalogCache.models;
|
||||
}
|
||||
|
||||
function storeCodexGithubCatalogCache(
|
||||
models: CodexDiscoveryModel[],
|
||||
response: Response,
|
||||
now: number,
|
||||
cacheTtlMs: number
|
||||
): void {
|
||||
const etag = toNonEmptyString(response.headers.get("etag"));
|
||||
codexGithubCatalogCache = {
|
||||
models,
|
||||
...(etag ? { etag } : {}),
|
||||
expiresAt: now + cacheTtlMs,
|
||||
};
|
||||
}
|
||||
|
||||
type CodexLocalCatalogModel = {
|
||||
id: string;
|
||||
name?: string;
|
||||
apiFormat?: string;
|
||||
supportedEndpoints?: string[];
|
||||
contextLength?: number;
|
||||
maxInputTokens?: number;
|
||||
maxOutputTokens?: number;
|
||||
};
|
||||
|
||||
function localCatalogModelToCodexDiscoveryModel(
|
||||
model: CodexLocalCatalogModel
|
||||
): CodexDiscoveryModel {
|
||||
const inputTokenLimit = firstPositiveNumber(model.maxInputTokens, model.contextLength);
|
||||
const outputTokenLimit = firstPositiveNumber(model.maxOutputTokens);
|
||||
return {
|
||||
id: model.id,
|
||||
name: model.name || model.id,
|
||||
owned_by: "codex",
|
||||
apiFormat: "responses",
|
||||
supportedEndpoints: ["responses"],
|
||||
...(typeof inputTokenLimit === "number" ? { inputTokenLimit } : {}),
|
||||
...(typeof outputTokenLimit === "number" ? { outputTokenLimit } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
export function mergeCodexLiveModelsWithLocalCatalog(
|
||||
liveModels: CodexDiscoveryModel[],
|
||||
localCatalogModels: CodexLocalCatalogModel[]
|
||||
): CodexDiscoveryModel[] {
|
||||
const merged = new Map<string, CodexDiscoveryModel>();
|
||||
|
||||
for (const liveModel of liveModels) {
|
||||
merged.set(liveModel.id, liveModel);
|
||||
}
|
||||
|
||||
for (const localModel of localCatalogModels) {
|
||||
if (!localModel.id) continue;
|
||||
const normalizedLocal = localCatalogModelToCodexDiscoveryModel(localModel);
|
||||
const existing = merged.get(localModel.id);
|
||||
merged.set(localModel.id, existing ? { ...normalizedLocal, ...existing } : normalizedLocal);
|
||||
}
|
||||
|
||||
return Array.from(merged.values());
|
||||
}
|
||||
|
||||
export function enrichCodexModelsFromGithubCatalog(
|
||||
models: CodexDiscoveryModel[],
|
||||
githubCatalogModels: CodexDiscoveryModel[]
|
||||
): CodexDiscoveryModel[] {
|
||||
const byId = new Map(githubCatalogModels.map((model) => [model.id, model]));
|
||||
return models.map((model) => {
|
||||
const githubModel = byId.get(model.id);
|
||||
return githubModel ? { ...githubModel, ...model } : model;
|
||||
});
|
||||
}
|
||||
|
||||
export async function fetchCodexDiscoveryModels({
|
||||
accessToken,
|
||||
providerSpecificData,
|
||||
fetchImpl,
|
||||
}: {
|
||||
accessToken: string | null;
|
||||
providerSpecificData?: Record<string, unknown> | null;
|
||||
fetchImpl: CodexModelsFetch;
|
||||
}): Promise<CodexDiscoveryModel[] | null> {
|
||||
if (!accessToken) return null;
|
||||
|
||||
try {
|
||||
const workspaceId =
|
||||
toNonEmptyString(providerSpecificData?.workspaceId) ||
|
||||
toNonEmptyString(providerSpecificData?.chatgptAccountId) ||
|
||||
toNonEmptyString(providerSpecificData?.accountId);
|
||||
const headers: Record<string, string> = {
|
||||
...getCodexDefaultHeaders(),
|
||||
Accept: "application/json",
|
||||
"Content-Type": "application/json",
|
||||
Authorization: `Bearer ${accessToken}`,
|
||||
originator: "codex_cli_rs",
|
||||
};
|
||||
if (workspaceId) headers["chatgpt-account-id"] = workspaceId;
|
||||
|
||||
const response = await fetchImpl(buildCodexModelsUrl(), {
|
||||
method: "GET",
|
||||
headers,
|
||||
});
|
||||
|
||||
if (!response.ok) return null;
|
||||
|
||||
const models = normalizeCodexModelsResponse(await response.json());
|
||||
return models.length > 0 ? models : null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
export async function fetchCodexGithubCatalogModels({
|
||||
fetchImpl,
|
||||
now = Date.now(),
|
||||
cacheTtlMs = CODEX_GITHUB_CATALOG_CACHE_TTL_MS,
|
||||
}: {
|
||||
fetchImpl: CodexModelsFetch;
|
||||
now?: number;
|
||||
cacheTtlMs?: number;
|
||||
}): Promise<CodexDiscoveryModel[] | null> {
|
||||
const cachedModels = getFreshCodexGithubCatalogCache(now, cacheTtlMs);
|
||||
if (cachedModels) return cachedModels;
|
||||
|
||||
try {
|
||||
const response = await fetchImpl(CODEX_GITHUB_MODELS_URL, {
|
||||
method: "GET",
|
||||
headers: buildCodexGithubCatalogHeaders(),
|
||||
});
|
||||
|
||||
const notModifiedModels = getNotModifiedCodexGithubCatalog(response, now, cacheTtlMs);
|
||||
if (notModifiedModels) return notModifiedModels;
|
||||
|
||||
if (!response.ok) return null;
|
||||
|
||||
const models = normalizeCodexGithubCatalogResponse(await response.json());
|
||||
if (models.length === 0) return null;
|
||||
|
||||
storeCodexGithubCatalogCache(models, response, now, cacheTtlMs);
|
||||
return models;
|
||||
} catch {
|
||||
return codexGithubCatalogCache?.models || null;
|
||||
}
|
||||
}
|
||||
@@ -97,6 +97,12 @@ import {
|
||||
type ProviderModelsConfigEntry,
|
||||
PROVIDER_MODELS_CONFIG,
|
||||
} from "./discovery/providerModelsConfig";
|
||||
import {
|
||||
enrichCodexModelsFromGithubCatalog,
|
||||
fetchCodexDiscoveryModels,
|
||||
fetchCodexGithubCatalogModels,
|
||||
mergeCodexLiveModelsWithLocalCatalog,
|
||||
} from "./discovery/codex";
|
||||
|
||||
/**
|
||||
* GET /api/providers/[id]/models - Get models list from provider
|
||||
@@ -330,14 +336,16 @@ export async function GET(
|
||||
const buildDiscoveryFallbackResponse = ({
|
||||
cacheWarning = "API unavailable — using cached catalog",
|
||||
localWarning = "API unavailable — using local catalog",
|
||||
localIntentional = false,
|
||||
}: {
|
||||
cacheWarning?: string;
|
||||
localWarning?: string;
|
||||
localIntentional?: boolean;
|
||||
} = {}) => {
|
||||
if (cachedDiscoveryModels.length > 0) {
|
||||
return buildCachedDiscoveryResponse(cacheWarning);
|
||||
}
|
||||
return buildLocalCatalogResponse(localWarning);
|
||||
return buildLocalCatalogResponse(localWarning, localIntentional);
|
||||
};
|
||||
|
||||
const buildDiscoveryErrorFallbackResponse = (
|
||||
@@ -1712,6 +1720,71 @@ export async function GET(
|
||||
});
|
||||
}
|
||||
|
||||
if (provider === "codex") {
|
||||
const cachedResponse = maybeReturnCachedDiscovery();
|
||||
if (cachedResponse) return cachedResponse;
|
||||
|
||||
const autoFetchDisabledResponse = maybeReturnAutoFetchDisabled();
|
||||
if (autoFetchDisabledResponse) return autoFetchDisabledResponse;
|
||||
|
||||
const liveModels = await fetchCodexDiscoveryModels({
|
||||
accessToken: accessToken || null,
|
||||
providerSpecificData: connection.providerSpecificData,
|
||||
fetchImpl: (url, init) =>
|
||||
safeOutboundFetch(url, {
|
||||
...SAFE_OUTBOUND_FETCH_PRESETS.modelsDiscovery,
|
||||
guard: getProviderOutboundGuard(),
|
||||
proxyConfig: proxy,
|
||||
...init,
|
||||
}),
|
||||
});
|
||||
const githubCatalogModels = await fetchCodexGithubCatalogModels({
|
||||
fetchImpl: (url, init) =>
|
||||
safeOutboundFetch(url, {
|
||||
...SAFE_OUTBOUND_FETCH_PRESETS.modelsDiscovery,
|
||||
guard: "public-only",
|
||||
proxyConfig: proxy,
|
||||
...init,
|
||||
}),
|
||||
});
|
||||
const staticCodexCatalog = mergeLocalCatalogModels(
|
||||
getModelsByProviderId("codex") || [],
|
||||
getStaticModelsForProvider("codex") || []
|
||||
);
|
||||
|
||||
if (liveModels && liveModels.length > 0) {
|
||||
const enrichedLiveModels =
|
||||
githubCatalogModels && githubCatalogModels.length > 0
|
||||
? enrichCodexModelsFromGithubCatalog(liveModels, githubCatalogModels)
|
||||
: liveModels;
|
||||
return buildApiDiscoveryResponse(
|
||||
mergeCodexLiveModelsWithLocalCatalog(enrichedLiveModels, staticCodexCatalog)
|
||||
);
|
||||
}
|
||||
|
||||
if (githubCatalogModels && githubCatalogModels.length > 0) {
|
||||
return buildApiDiscoveryResponse(
|
||||
mergeCodexLiveModelsWithLocalCatalog(githubCatalogModels, staticCodexCatalog),
|
||||
"Codex live catalog unavailable — using GitHub model catalog"
|
||||
);
|
||||
}
|
||||
|
||||
const fallback = buildDiscoveryFallbackResponse({
|
||||
cacheWarning: "Codex live catalog unavailable — using cached catalog",
|
||||
localWarning: "Codex live and GitHub catalogs unavailable — using local catalog",
|
||||
localIntentional: true,
|
||||
});
|
||||
if (fallback) return fallback;
|
||||
return buildResponse({
|
||||
provider,
|
||||
connectionId,
|
||||
models: [],
|
||||
source: "local_catalog",
|
||||
intentional: true,
|
||||
warning: "Codex live and GitHub catalogs unavailable — using local catalog",
|
||||
});
|
||||
}
|
||||
|
||||
const localCatalog = mergeLocalCatalogModels(registryCatalogModels, specialtyCatalogModels);
|
||||
if (!config && localCatalog.length > 0) {
|
||||
return buildResponse({
|
||||
|
||||
@@ -25,6 +25,7 @@
|
||||
"noData": "Nenhum dado disponível",
|
||||
"nothingHere": "Nothing here yet",
|
||||
"configure": "Configurar",
|
||||
"manualConfig": "Configuração manual",
|
||||
"manage": "Gerenciar",
|
||||
"name": "Nome",
|
||||
"actions": "Ações",
|
||||
@@ -33,6 +34,7 @@
|
||||
"model": "Modelo",
|
||||
"models": "modelos",
|
||||
"provider": "Provedor",
|
||||
"unknownProvider": "Provedor desconhecido",
|
||||
"account": "Conta",
|
||||
"time": "Tempo",
|
||||
"details": "Detalhes",
|
||||
@@ -843,6 +845,7 @@
|
||||
"batchActionRetryError": "Falha ao refazer requisições com falha. Tente novamente.",
|
||||
"batchConceptRetentionNote": "Arquivos de resultado e erro ficam retidos por 30 dias (Anthropic: 29 dias)"
|
||||
},
|
||||
"disabled": "Desabilitado",
|
||||
"featureFlagOmnirouteEmergencyFallbackDescription": "__MISSING__:Route budget-exhausted requests to the emergency free fallback provider/model.",
|
||||
"featureFlagArenaEloSyncEnabledDescription": "__MISSING__:Enable periodic Arena AI leaderboard ELO sync for model intelligence rankings.",
|
||||
"sidebar": {
|
||||
@@ -1115,7 +1118,10 @@
|
||||
"dragReorderItem": "__MISSING__:Drag to reorder",
|
||||
"cannotHide": "__MISSING__:This item cannot be hidden",
|
||||
"alwaysVisible": "__MISSING__:Always visible",
|
||||
"groupSeparatorLabel": "__MISSING__:Separator"
|
||||
"groupSeparatorLabel": "__MISSING__:Separator",
|
||||
"discovery": "Descoberta",
|
||||
"discoverySubtitle": "Buscar acesso gratuito em provedores",
|
||||
"chaosConfig": "Chaos Mode"
|
||||
},
|
||||
"webhooks": {
|
||||
"title": "Webhooks",
|
||||
@@ -1761,6 +1767,8 @@
|
||||
"copyMaskedKey": "Copiar chave mascarada",
|
||||
"keyOnlyAvailableAtCreation": "Full key available only at creation time — copy it when you first create the key",
|
||||
"modelsCount": "{count, plural, one {# modelo} other {# modelos}}",
|
||||
"devicesCount": "{count, plural, one {# dispositivo} other {# dispositivos}}",
|
||||
"devicesTooltip": "{count, plural, one {# dispositivo distinto (IP/User-Agent) visto com esta chave (últimos 30 min)} other {# dispositivos distintos (IP/User-Agent) vistos com esta chave (últimos 30 min)}}",
|
||||
"lastUsedOn": "Último: {date}",
|
||||
"viewCostsFor": "__MISSING__:View costs for {name}",
|
||||
"editPermissions": "Editar permissões",
|
||||
@@ -1773,6 +1781,8 @@
|
||||
"models": "{count} modelos",
|
||||
"permissionsTitle": "Permissões: {name}",
|
||||
"allowAllDesc": "Esta chave pode acessar todos os modelos disponíveis.",
|
||||
"restrictLoading": "Carregando catálogo de modelos…",
|
||||
"restrictCatalogUnavailable": "Catálogo de modelos indisponível; esta chave tem {selectedCount} restrições de modelo selecionadas.",
|
||||
"restrictDesc": "Esta chave pode acessar {selectedCount} de {totalModels} modelos.",
|
||||
"selectedCount": "{count} selecionados",
|
||||
"maxActiveSessions": "Máximo de sessões ativas",
|
||||
@@ -1856,14 +1866,16 @@
|
||||
"webSearch": "Busca Web",
|
||||
"webFetch": "Web Fetch",
|
||||
"video": "Vídeo",
|
||||
"music": "Música"
|
||||
"music": "Música",
|
||||
"ocr": "OCR"
|
||||
},
|
||||
"noProviders": "Nenhum provedor configurado para este tipo.",
|
||||
"addConnection": "Adicionar Conexão",
|
||||
"backToProviders": "Voltar para Provedores",
|
||||
"connections": "{count} Conexões",
|
||||
"noConnections": "Nenhuma conexão ainda — adicione uma na página do provedor.",
|
||||
"loading": "Carregando..."
|
||||
"loading": "Carregando...",
|
||||
"suggestedModels": "Modelos sugeridos pelo provedor"
|
||||
},
|
||||
"search": {
|
||||
"searchQuery": "Search Query",
|
||||
@@ -2391,6 +2403,14 @@
|
||||
"strictRandomDesc": "Baralho embaralhado — usa cada modelo uma vez antes de reembaralhar",
|
||||
"fusion": "__MISSING__:Fusion",
|
||||
"fusionDesc": "__MISSING__:Fans the prompt to every panel model in parallel, then a judge model synthesizes one final answer",
|
||||
"fusionJudgeModel": "Modelo julgador",
|
||||
"fusionJudgeModelHelp": "Modelo que sintetiza as respostas do painel em uma resposta final única. Deixe vazio para usar o primeiro modelo do painel.",
|
||||
"fusionMinPanel": "Painel mínimo",
|
||||
"fusionMinPanelHelp": "Respostas bem-sucedidas do painel necessárias antes que os retardatários recebam uma janela de tolerância (padrão 2).",
|
||||
"fusionStragglerGraceMs": "Tolerância para retardatários (ms)",
|
||||
"fusionStragglerGraceMsHelp": "Quanto tempo esperar por modelos lentos do painel após o quórum ser atingido (padrão 8000).",
|
||||
"fusionPanelHardTimeoutMs": "Timeout máximo do painel (ms)",
|
||||
"fusionPanelHardTimeoutMsHelp": "Limite absoluto para que um modelo travado não bloqueie o painel inteiro (padrão 90000).",
|
||||
"models": "Modelos",
|
||||
"autoBalance": "Auto-balancear",
|
||||
"advancedSettings": "Configurações Avançadas",
|
||||
@@ -2500,7 +2520,8 @@
|
||||
"queueTimeout": "Tempo máximo em fila antes de expirar no round-robin.",
|
||||
"failoverBeforeRetry": "Quando ativado, qualquer erro upstream aciona o failover imediato para o próximo destino combinado, ignorando todas as novas tentativas e URLs substitutos.",
|
||||
"maxSetRetries": "Número de vezes para tentar novamente o destino completo definido quando todos os alvos falharem. 0 = nenhuma nova tentativa de nível definido.",
|
||||
"setRetryDelayMs": "Atraso entre novas tentativas de nível definido, dando tempo para que problemas transitórios sejam resolvidos."
|
||||
"setRetryDelayMs": "Atraso entre novas tentativas de nível definido, dando tempo para que problemas transitórios sejam resolvidos.",
|
||||
"disableSessionStickiness": "Alterna para uma conexão diferente a cada requisição, em vez de fixar toda a conversa em uma conexão pelo hash da primeira mensagem. Sobrescreve o padrão global. Deixe em Herdar para preservar acertos de cache de prompt em conversas com múltiplos turnos."
|
||||
},
|
||||
"templatesTitle": "Templates rápidos",
|
||||
"templatesDescription": "Aplique um perfil inicial e depois ajuste modelos e configuração.",
|
||||
@@ -2794,6 +2815,13 @@
|
||||
"browseLegacyCatalog": "Navegue pelo catálogo combinado legado",
|
||||
"agentFeaturesTitle": "Recursos do agente",
|
||||
"agentFeaturesDescription": "Habilite recursos avançados para agentes usando esta combinação",
|
||||
"responseValidationTitle": "Validação de resposta",
|
||||
"responseValidationHelp": "Faz failover para o próximo alvo quando um corpo 200 OK falha nestas verificações (conteúdo do assistente).",
|
||||
"responseValidationForbidden": "Substrings proibidas (uma por linha)",
|
||||
"responseValidationRequired": "Substrings obrigatórias (uma por linha)",
|
||||
"responseValidationMinLength": "Tamanho mínimo do conteúdo (caracteres)",
|
||||
"responseValidationJsonPaths": "Verificações de JSON-path",
|
||||
"responseValidationAddCheck": "+ Adicionar verificação",
|
||||
"agentFeaturesSystemMessageOverride": "Substituir mensagem do sistema",
|
||||
"agentFeaturesSystemMessagePlaceholder": "Você é um assistente especialista...",
|
||||
"agentFeaturesSystemMessageHint": "Substituição de mensagens do sistema para agentes",
|
||||
@@ -2807,7 +2835,10 @@
|
||||
"agentFeaturesContextLengthErrorInteger": "O comprimento do contexto precisa ser um número inteiro válido",
|
||||
"agentFeaturesContextLengthErrorRange": "O comprimento do contexto precisa estar entre 1.000 e 2.000.000",
|
||||
"compressionOverride": "Substituição de compactação",
|
||||
"modePack": "Pacote de modos"
|
||||
"modePack": "Pacote de modos",
|
||||
"disableSessionStickiness": "Desabilitar fixação de sessão",
|
||||
"sessionStickinessEnabled": "Fixação ativada",
|
||||
"sessionStickinessDisabled": "Fixação desativada"
|
||||
},
|
||||
"costs": {
|
||||
"title": "Custos",
|
||||
@@ -3410,6 +3441,10 @@
|
||||
"description": "Integração opcional com Qdrant para busca semântica escalável",
|
||||
"enableLabel": "Habilitar Qdrant",
|
||||
"enableDesc": "Quando habilitado, Qdrant é usado como vector store principal",
|
||||
"banner": "Armazenamento vetorial Tier 2 — uma alternativa externa e escalável ao sqlite-vec embutido (Tier 1). Habilite apenas se você tiver um conjunto de memória muito grande ou quiser memória compartilhada entre instâncias; a maioria dos usuários fica bem com o sqlite-vec. Quando habilitado, ele se torna o armazenamento primário e volta automaticamente para o sqlite-vec se ficar inacessível.",
|
||||
"hostHelp": "Docker local: http://localhost:6333 · Qdrant Cloud: a URL do seu cluster",
|
||||
"collectionHelp": "Qualquer nome — o OmniRoute o cria no primeiro uso",
|
||||
"embeddingModelHelp": "Define a dimensão do vetor automaticamente no primeiro uso. Memórias existentes não são retroalimentadas, e trocar o modelo depois que já existem dados exige uma coleção nova.",
|
||||
"testConnection": "Testar conexão",
|
||||
"testing": "Testando...",
|
||||
"statusActive": "Ativo",
|
||||
@@ -3486,6 +3521,7 @@
|
||||
"noExecutions": "Nenhuma execução encontrada",
|
||||
"enabled": "Ativado",
|
||||
"disabled": "Desativado",
|
||||
"delete": "Excluir",
|
||||
"version": "Versão",
|
||||
"tableDescription": "Descrição",
|
||||
"skill": "Skill",
|
||||
@@ -4065,6 +4101,8 @@
|
||||
"prefixHint": "Obrigatório. Prefixo único para nomes de modelos.",
|
||||
"nameHint": "Obrigatório. Um rótulo amigável para este nó.",
|
||||
"baseUrlHint": "Obrigatório. URL base da API do provedor.",
|
||||
"iconUrlLabel": "URL do ícone",
|
||||
"iconUrlHint": "Opcional. URL da imagem exibida como ícone deste provedor.",
|
||||
"anthropicPrefixPlaceholder": "ac-prod",
|
||||
"openaiPrefixPlaceholder": "oc-prod",
|
||||
"anthropicBaseUrlPlaceholder": "https://api.anthropic.com/v1",
|
||||
@@ -4176,6 +4214,8 @@
|
||||
"optional": "Opcional",
|
||||
"anthropicCompatibleName": "Compatível Anthropic",
|
||||
"openaiCompatibleName": "Compatível OpenAI",
|
||||
"compatibleDefaultModelLabel": "Modelo padrão",
|
||||
"compatibleDefaultModelHint": "Digite o ID do modelo exatamente como seu endpoint compatível espera. Este modelo será salvo como o padrão da conexão.",
|
||||
"failedImportModels": "Falha ao importar modelos",
|
||||
"noModelsReturnedFromEndpoint": "Nenhum modelo retornado do endpoint /models.",
|
||||
"importingModelsProgress": "Importando {current} de {total} modelos...",
|
||||
@@ -4282,10 +4322,29 @@
|
||||
"targetFormatAuto": "__MISSING__:Default (auto)",
|
||||
"targetFormatGemini": "__MISSING__:Gemini",
|
||||
"targetFormatAntigravity": "__MISSING__:Antigravity",
|
||||
"contextWindowOverrideLabel": "__MISSING__:Context Window Override",
|
||||
"contextWindowOverridePlaceholder": "__MISSING__:e.g. 131072",
|
||||
"contextWindowOverrideHint": "__MISSING__:Manually set this model's real context window (tokens) when the provider misreports it. Wins over auto-detected/catalog values and prevents combo routing from dropping the model.",
|
||||
"contextWindowOverrideInvalid": "__MISSING__:Context window override must be a positive whole number of tokens",
|
||||
"compatParamFiltersLabel": "Filtros de parâmetros",
|
||||
"compatBlockedParamsHint": "Parâmetros bloqueados (removidos das requisições)",
|
||||
"compatAllowedParamsHint": "Parâmetros permitidos (readicionados após o bloqueio)",
|
||||
"paramFiltersSectionTitle": "Filtros de parâmetros",
|
||||
"paramFiltersSectionHint": "Remove ou readiciona parâmetros da requisição antes de enviá-la a este provedor. Usado para evitar erros 400 de provedores que rejeitam certos parâmetros (ex.: NVIDIA NIM rejeita <code>thinking</code>).",
|
||||
"paramFiltersBlockedLabel": "Parâmetros bloqueados",
|
||||
"paramFiltersBlockedHint": "Estes parâmetros são removidos das requisições de saída (lista de bloqueio).",
|
||||
"paramFiltersAllowedLabel": "Parâmetros permitidos",
|
||||
"paramFiltersAllowedHint": "Estes parâmetros são readicionados após a remoção pela lista de bloqueio (somente se o cliente os enviou).",
|
||||
"paramFiltersAutoLearnLabel": "Aprender automaticamente com erros 400",
|
||||
"paramFiltersAutoLearnHint": "Quando habilitado, se o upstream retornar 400 com \"Unsupported parameter: X\", o parâmetro é automaticamente adicionado à lista de bloqueio e a requisição é refeita.",
|
||||
"paramFiltersSaving": "Salvando…",
|
||||
"paramFiltersSaveChanges": "Salvar alterações",
|
||||
"paramFiltersResetToDefault": "Restaurar padrão",
|
||||
"paramFiltersLoadError": "Falha ao carregar configuração de filtro de parâmetros: {error}",
|
||||
"paramFiltersSaveSuccess": "Configuração de filtro de parâmetros salva",
|
||||
"paramFiltersSaveError": "Falha ao salvar configuração de filtro de parâmetros: {error}",
|
||||
"paramFiltersResetSuccess": "Configuração de filtro de parâmetros restaurada ao padrão",
|
||||
"paramFiltersResetError": "Falha ao restaurar configuração de filtro de parâmetros: {error}",
|
||||
"contextWindowOverrideLabel": "Substituição de Janela de Contexto",
|
||||
"contextWindowOverridePlaceholder": "ex.: 131072",
|
||||
"contextWindowOverrideHint": "Define manualmente a janela de contexto real (tokens) deste modelo quando o provedor a reporta incorretamente. Tem prioridade sobre os valores detectados automaticamente/do catálogo e evita que o roteamento de combo descarte o modelo.",
|
||||
"contextWindowOverrideInvalid": "A substituição de janela de contexto deve ser um número inteiro positivo de tokens",
|
||||
"compatUpstreamHeadersLabel": "Extra upstream headers",
|
||||
"compatUpstreamHeadersHint": "High-privilege setting — same trust level as editing provider API credentials; only trusted admins should use it. Merged after OmniRoute adds auth from the provider API key. If a custom header uses the same name as an existing one (e.g. Authorization), your value fully replaces the auto-generated header (including the Bearer token) — the upstream only sees what you typed, not the key from settings. Misconfiguration can cause 401 or broken upstream auth. One row per header (e.g. extra Authentication for some gateways). Hover or focus the value to preview. Saves on blur, outside click, or closing this panel.",
|
||||
"compatUpstreamHeaderName": "Header name",
|
||||
@@ -4482,6 +4541,8 @@
|
||||
"modelsPathLabel": "Models Endpoint Path",
|
||||
"modelsPathPlaceholder": "/models",
|
||||
"modelsPathHint": "Custom models path for validation (e.g. /v4/models)",
|
||||
"clientIdentityLabel": "Identidade do cliente",
|
||||
"clientIdentityHint": "Opcional. Adiciona cabeçalhos de fingerprint do cliente (ex.: User-Agent) correspondentes a um CLI conhecido, para gateways compatíveis que esperam um.",
|
||||
"statusDeactivated": "Desativado (Manual)",
|
||||
"statusBanned": "Banido / Sandbox Violation",
|
||||
"statusCreditsExhausted": "Saldo Insuficiente",
|
||||
@@ -4604,6 +4665,11 @@
|
||||
"localProviderBaseUrlHint": "Obrigatório: cole a URL base OpenAI-compatible /v1 do seu {provider} (padrão: {baseUrl}). URLs locais/privadas exigem OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS=true para validação no dashboard.",
|
||||
"localProviders": "Local / Self-Hosted",
|
||||
"maxConcurrentWholeNumberError": "Max concurrent deve ser um número inteiro maior ou igual a 0.",
|
||||
"m365TierLabel": "Nível do Copilot",
|
||||
"m365TierHint": "Escolha qual superfície do Microsoft 365 Copilot esta conexão usa. Individual é o BizChat padrão para consumidores; Educação e Empresarial (trabalho) usam as superfícies do respectivo tenant.",
|
||||
"m365TierIndividualOption": "Individual (padrão)",
|
||||
"m365TierEduOption": "Educação",
|
||||
"m365TierEnterpriseOption": "Empresarial / Trabalho",
|
||||
"museSparkWebCookieHint": "Cole o cookie abra_sess do meta.ai. Um cabeçalho de cookie completo também funciona.",
|
||||
"museSparkWebCookiePlaceholder": "Cole o valor de abra_sess",
|
||||
"oauth": "Oauth",
|
||||
@@ -4787,6 +4853,7 @@
|
||||
"bedrockModelsDescription": "Os modelos Amazon Bedrock têm escopo definido por região da AWS. Importe de /models ou adicione IDs de modelo Bedrock habilitados na região selecionada.",
|
||||
"bedrockModelPlaceholder": "antrópico.claude-sonnet-4-6",
|
||||
"addProviderSessionCookieTitle": "Adicionar cookie de sessão {provider}",
|
||||
"openWebProviderSite": "Abrir {host}",
|
||||
"addProviderWebTokenTitle": "Adicionar token da web {provider}",
|
||||
"addProviderConnectionTitle": "Adicionar conexão {provider}",
|
||||
"webTokenCredentialLabel": "Token de sessão da Web",
|
||||
@@ -4820,7 +4887,11 @@
|
||||
"kimiWebLabel": "__MISSING__:Kimi Web",
|
||||
"kimiWebDesc": "__MISSING__:Moonshot AI consumer chat via www.kimi.com (international, Connect-RPC API)",
|
||||
"doubaoWebLabel": "__MISSING__:Doubao Web",
|
||||
"doubaoWebDesc": "__MISSING__:ByteDance AI chat via doubao.com"
|
||||
"doubaoWebDesc": "__MISSING__:ByteDance AI chat via doubao.com",
|
||||
"overrideBaseUrlAdvanced": "Avançado: sobrescrever URL base",
|
||||
"overrideBaseUrlHint": "Avançado: aponta este provedor embutido para um endpoint personalizado. Deixe em branco para usar o padrão.",
|
||||
"bulkAddFormatHintCloudflare": "Uma chave por linha. Formato: nome|accountId|apiKey (ID de conta Cloudflare + token de API).",
|
||||
"lmarenaWebCookieHint": "Abra arena.ai, faça login e depois copie o cabeçalho Cookie completo de uma requisição de rede. Inclua arena-auth-prod-v1.0 e arena-auth-prod-v1.1 (e outros fragmentos, se houver), preferencialmente com cf_clearance. Não cole apenas o cookie vazio arena-auth-prod-v1. Opcional: providerSpecificData.recaptchaV3Token se create-evaluation ainda retornar 403."
|
||||
},
|
||||
"settings": {
|
||||
"title": "Configurações",
|
||||
@@ -4830,6 +4901,10 @@
|
||||
"routing": "Roteamento",
|
||||
"cache": "Cache",
|
||||
"resilience": "Resiliência",
|
||||
"description": "Descrição",
|
||||
"enable": "Habilitar",
|
||||
"disable": "Desabilitar",
|
||||
"update": "Atualizar",
|
||||
"routingSettingsIntro": "Controla como suas solicitações são roteadas, transformadas e enviadas aos provedores de IA.",
|
||||
"routingOpDropParagraphContainsLabel": "Remover parágrafo (contém)",
|
||||
"routingOpDropParagraphStartsWithLabel": "Remover parágrafo (começa com)",
|
||||
@@ -4965,6 +5040,8 @@
|
||||
"modelsDevInfoOrder": "User Override → models.dev → LiteLLM → Hardcoded Default",
|
||||
"systemTheme": "Tema do Sistema",
|
||||
"debugToggle": "Enable Debug Mode",
|
||||
"logToolSourcesToggle": "Registrar fontes de ferramentas",
|
||||
"logToolSourcesDescription": "Emite uma linha de log de diagnóstico por requisição resumindo a contagem de ferramentas e a origem MCP/hospedada/cliente.",
|
||||
"homePinProviderQuotaToHome": "Fixar informações na página inicial",
|
||||
"homeProviderQuotaLimits": "Limites de cota dos provedores",
|
||||
"homeProviderQuotaLimitsDesc": "Fixe o painel de status de cotas dos provedores (com botão Atualizar tudo) no topo da página inicial.",
|
||||
@@ -5358,6 +5435,9 @@
|
||||
"providerMaxRetriesAria": "{provider} tentativas máximas",
|
||||
"providerTimeoutAria": "timeout de {provider} em ms",
|
||||
"removeProviderOverrideAria": "Remover sobrescrita de {provider}",
|
||||
"selectProviderPlaceholder": "Selecionar provedor...",
|
||||
"searchProviderPlaceholder": "Buscar provedores...",
|
||||
"searchProviderAria": "Buscar provedores",
|
||||
"newProviderNamePlaceholder": "ex.: google, openai...",
|
||||
"newProviderNameAria": "Nome do novo provedor",
|
||||
"retries": "tentativas",
|
||||
@@ -5366,6 +5446,7 @@
|
||||
"maxNestingDepth": "Profundidade Máx. de Aninhamento",
|
||||
"concurrencyPerModel": "Concorrência / Modelo",
|
||||
"queueTimeout": "Timeout da Fila (ms)",
|
||||
"queueDepth": "Profundidade da fila",
|
||||
"contextRelayHandoffThreshold": "Limite de Handoff",
|
||||
"contextRelayMaxMessages": "Máx. de Mensagens no Resumo",
|
||||
"contextRelaySummaryModel": "Modelo do Resumo",
|
||||
@@ -5519,6 +5600,22 @@
|
||||
"maintenance": "Maintenance",
|
||||
"purgeExpiredLogs": "Purgar Logs Expirados",
|
||||
"purgeLogsFailed": "Falha ao purgar logs",
|
||||
"logsDeleted": "{count, plural, =0 {Nenhum log expirado removido} one {# log expirado removido} other {# logs expirados removidos}}",
|
||||
"resetUsageData": "Redefinir dados de uso",
|
||||
"resetUsageDataDesc": "Selecione quanto tempo no passado você deseja excluir dados de uso. Esta ação não pode ser desfeita.",
|
||||
"resetUsagePeriod_5m": "5 minutos",
|
||||
"resetUsagePeriod_1h": "1 hora",
|
||||
"resetUsagePeriod_3h": "3 horas",
|
||||
"resetUsagePeriod_6h": "6 horas",
|
||||
"resetUsagePeriod_12h": "12 horas",
|
||||
"resetUsagePeriod_1d": "1 dia",
|
||||
"resetUsagePeriod_7d": "7 dias",
|
||||
"resetUsagePeriod_30d": "30 dias",
|
||||
"resetUsagePeriod_all": "Todo o período",
|
||||
"resetUsageSuccess": "{count, plural, =0 {Nenhuma linha de dados de uso excluída} one {Dados de uso redefinidos (# linha excluída)} other {Dados de uso redefinidos (# linhas excluídas)}}",
|
||||
"resetUsageFailed": "Falha ao redefinir dados de uso",
|
||||
"reset": "Redefinir",
|
||||
"resetting": "Redefinindo...",
|
||||
"contextOpt": "Context Optimized",
|
||||
"contextOptDesc": "Routes based on context window requirements and conversation length",
|
||||
"priorityDesc": "Sequential fallback - tries provider 1 first, then provider 2, and so on",
|
||||
@@ -5619,8 +5716,12 @@
|
||||
"compressionAutoTrigger": "Auto-Trigger Threshold",
|
||||
"compressionCacheTTL": "Cache TTL",
|
||||
"compressionPreserveSystem": "Preserve System Prompt",
|
||||
"compressionPreserveSystemAlways": "Sempre",
|
||||
"compressionPreserveSystemWhenNoCache": "Quando não há cache",
|
||||
"compressionPreserveSystemNever": "Nunca",
|
||||
"compressionCavemanConfig": "Caveman Engine Configuration",
|
||||
"compressionCavemanConfigDesc": "Fine-tune the rule-based compression engine",
|
||||
"compressionCavemanPanelHint": "Ativado/desativado e o nível são definidos no painel:",
|
||||
"compressionRoles": "Compress Message Roles",
|
||||
"compressionRoleUser": "User",
|
||||
"compressionRoleAssistant": "Assistant",
|
||||
@@ -5827,6 +5928,17 @@
|
||||
"storageUsageTokenBufferDesc": "__MISSING__:Extra tokens added to reported usage to account for system prompt overhead.",
|
||||
"storageUsageTokenBufferHint": "__MISSING__:Set to 0 to report raw provider token counts. Default: 2000.",
|
||||
"storageUsageTokenBufferCurrent": "__MISSING__:Current: {value}",
|
||||
"redisLauncherTitle": "Redis local",
|
||||
"redisLauncherDesc": "Inicie com um clique um contêiner Redis 7 (Podman ou Docker) para cache de resposta, controle de cota e limitação de taxa.",
|
||||
"redisLauncherRefresh": "Atualizar",
|
||||
"redisLauncherStop": "Parar",
|
||||
"redisLauncherLaunching": "Iniciando...",
|
||||
"redisLauncherLaunch": "Iniciar Redis",
|
||||
"redisLauncherContainer": "Contêiner",
|
||||
"redisLauncherRunning": "Em execução",
|
||||
"redisLauncherReachable": "Acessível",
|
||||
"redisLauncherError": "Erro: {message}",
|
||||
"redisLauncherHint": "Equivalente a executar `omniroute redis up`. O contêiner é chamado `omniroute-redis` e escuta em 127.0.0.1:6379.",
|
||||
"compressionSettingsAutoTriggerMode": "Modo de disparo automático",
|
||||
"compressionSettingsMcpDescriptionCompression": "Compressão de descrição MCP",
|
||||
"mcpAccessibilityTitle": "__MISSING__:MCP accessibility output",
|
||||
@@ -5910,6 +6022,12 @@
|
||||
"claudeFastModeModelCheckbox": "Ativar modo rápido para {model}",
|
||||
"claudeFastModeSaveError": "Falha ao atualizar a configuração do modo rápido do Claude",
|
||||
"authz": {
|
||||
"cors": {
|
||||
"wildcard": {
|
||||
"title": "CORS está aberto para qualquer origem (CORS_ALLOW_ALL=true)",
|
||||
"desc": "Qualquer site pode chamar a API deste servidor a partir do navegador de um visitante. Use apenas em redes confiáveis — defina origens explícitas em ALLOWED_ORIGINS e desabilite o CORS_ALLOW_ALL em produção."
|
||||
}
|
||||
},
|
||||
"title": "Inventário de Authz",
|
||||
"description": "Classificação de rotas em 5 níveis com política de bypass em tempo real. A leitura mostra a taxonomia completa; as mutações solicitam a senha de gerenciamento.",
|
||||
"loading": "Carregando inventário…",
|
||||
@@ -5998,6 +6116,14 @@
|
||||
"resilienceEnableServerWaitDesc": "Quando ativado, o OmniRoute aguarda o primeiro resfriamento expirar e tenta novamente automaticamente.",
|
||||
"resilienceMaxAttempts": "Máximo de tentativas",
|
||||
"resilienceMaxWaitPerAttempt": "Espera máxima por tentativa",
|
||||
"resilienceComboCooldownWaitTitle": "Espera de cooldown do combo de cota compartilhada",
|
||||
"resilienceComboCooldownWaitDesc": "Somente para combos de cota compartilhada: aguarda um cooldown transitório curto e reenvia em vez de retornar um 429 imediatamente. Nunca espera em quota_exhausted.",
|
||||
"resilienceComboCooldownWaitToggleDesc": "Somente combos de cota compartilhada; nunca espera em quota_exhausted.",
|
||||
"resilienceComboCooldownMaxWaitMs": "Espera máxima por tentativa",
|
||||
"resilienceComboCooldownBudgetMs": "Orçamento total de espera",
|
||||
"resilienceQuotaShareConcurrencyTitle": "Concorrência por conexão de cota compartilhada",
|
||||
"resilienceQuotaShareConcurrencyDesc": "Somente para combos de cota compartilhada: quando uma conexão define um limite de Concorrência Máxima, serializa as requisições concorrentes para essa conta de assinatura para que nunca seja inundada além do teto. Requisições excedentes esperam na fila em vez de receber um 429. O limite vem do campo Concorrência Máxima de cada conexão; este botão apenas habilita ou desabilita seu respeito.",
|
||||
"resilienceQuotaShareConcurrencyToggleDesc": "Somente combos de cota compartilhada; respeita o limite de Concorrência Máxima de cada conexão.",
|
||||
"resilienceProviderCooldownTitle": "__MISSING__:Provider Cooldown",
|
||||
"resilienceProviderCooldownScope": "__MISSING__:All combo requests",
|
||||
"resilienceProviderCooldownTrigger": "__MISSING__:When a provider/connection fails",
|
||||
@@ -6066,7 +6192,39 @@
|
||||
"modelLockoutExponentialBackoff": "__MISSING__:Exponential Backoff",
|
||||
"modelLockoutExponentialBackoffDescription": "__MISSING__:When enabled, each consecutive failure increases the cooldown duration exponentially.",
|
||||
"modelLockoutMaxBackoffSteps": "__MISSING__:Max Backoff Steps",
|
||||
"modelLockoutMaxBackoffStepsDescription": "__MISSING__:Maximum number of backoff steps before the cooldown stops growing. The Max Cooldown cap is reached first in most configurations, making this a safety ceiling for when Max Cooldown is raised."
|
||||
"modelLockoutMaxBackoffStepsDescription": "__MISSING__:Maximum number of backoff steps before the cooldown stops growing. The Max Cooldown cap is reached first in most configurations, making this a safety ceiling for when Max Cooldown is raised.",
|
||||
"disableSessionStickiness": "Desabilitar fixação de sessão",
|
||||
"disableSessionStickinessDesc": "Combos round-robin e aleatórios alternam para uma conexão diferente a cada requisição, em vez de fixar toda a conversa em uma conexão pelo hash da primeira mensagem. Deixe desativado para preservar acertos de cache de prompt em conversas com múltiplos turnos. Sobrescritas por combo têm prioridade.",
|
||||
"accountRoundRobin": "Round Robin",
|
||||
"accountRoundRobinDesc": "Alterna entre contas para distribuir a carga",
|
||||
"addKeyValue": "Adicionar valor da chave",
|
||||
"comboRoundRobin": "Round Robin de Combo",
|
||||
"comboRoundRobinDesc": "Alterna entre os alvos do combo em vez de sempre começar pelo primeiro",
|
||||
"comboStickyLimit": "Limite de Fixação do Combo",
|
||||
"comboStickyLimitDesc": "Chamadas por alvo do combo antes de trocar",
|
||||
"configured": "configurado",
|
||||
"modelOverrideLoadFailed": "Falha ao carregar substituições de modelo",
|
||||
"modelOverrideRemoveFailed": "Falha ao remover substituição de modelo",
|
||||
"modelOverrideRemoved": "Substituição de modelo removida",
|
||||
"modelOverrideSaveFailed": "Falha ao salvar substituição de modelo",
|
||||
"modelOverrideSaved": "Substituição de modelo salva",
|
||||
"modelOverrideValuePlaceholder": "Valor numérico",
|
||||
"modelOverridesDesc": "Substitui as capacidades de provedor/modelo usadas pelo roteamento e pela formatação de requisições. Os alvos usam o mesmo formato provedor/modelo dos combos.",
|
||||
"modelOverridesTitle": "Substituições de Modelo",
|
||||
"noModelOverrides": "Nenhuma substituição configurada para este modelo.",
|
||||
"none": "Nenhum",
|
||||
"providerAccountRoutingDesc": "Substitui a estratégia global de contas para este provedor (paridade com 9router).",
|
||||
"providerAccountRoutingTitle": "Roteamento multi-conta",
|
||||
"providerRoutingInheritGlobal": "Herdar padrão global",
|
||||
"providerRoutingStrategy": "Estratégia de conta",
|
||||
"routingStrategyAccountSummary": "Distribuindo requisições entre contas com {limit} chamadas por conta.",
|
||||
"routingStrategyComboFallbackSummary": " Combos usam a estratégia configurada de cada combo (prioridade/fallback padrão).",
|
||||
"routingStrategyComboSummary": " Combos giram após {limit} chamada(s) por alvo.",
|
||||
"routingStrategyFillFirstSummary": "Usando contas em ordem de prioridade (Fill First).",
|
||||
"routingStrategySubtitle": "Compatível com 9router: round-robin de contas, limites de fixação e rotação de combo",
|
||||
"routingStrategyTitle": "Estratégia de Roteamento",
|
||||
"searchModelOverrideTargets": "Buscar provedor/modelo...",
|
||||
"selectedModel": "Modelo selecionado"
|
||||
},
|
||||
"contextRtk": {
|
||||
"title": "Motor RTK",
|
||||
@@ -6639,6 +6797,8 @@
|
||||
"autoRefresh": "Atualização automática",
|
||||
"refreshAll": "Atualizar tudo",
|
||||
"loadingQuotas": "Carregando...",
|
||||
"showMoreQuotas": "Mostrar mais {count}",
|
||||
"showLessQuotas": "Mostrar menos",
|
||||
"account": "Conta",
|
||||
"modelQuotas": "Cotas de Modelo",
|
||||
"lastUsed": "Last Refreshed",
|
||||
@@ -6689,6 +6849,11 @@
|
||||
"resetsIn": "Reseta em",
|
||||
"editCutoffs": "Editar pontos de corte",
|
||||
"forceRefresh": "Atualizar agora",
|
||||
"resetCreditsLabel": "Créditos de redefinição",
|
||||
"redeemResetCredit": "Resgatar redefinição",
|
||||
"confirmRedeemResetCredit": "Resgatar um crédito de redefinição do Codex para esta conta? Isso consome um crédito de redefinição.",
|
||||
"resetCreditRedeemed": "Redefinição resgatada",
|
||||
"resetCreditRedeemFailed": "Falha ao resgatar crédito de redefinição",
|
||||
"suiteBuilderSaveFailed": "Falha ao salvar a suíte customizada",
|
||||
"clone": "Clonar",
|
||||
"exportSuite": "Exportar",
|
||||
@@ -7841,7 +8006,37 @@
|
||||
"bulkImportMaxExceeded": "Máximo de 100 proxies por importação",
|
||||
"bulkImportPreview": "Visualização",
|
||||
"clearAssignment": "(atribuição clara)",
|
||||
"bulkProxyAssignment": "Atribuição de proxy em massa"
|
||||
"bulkProxyAssignment": "Atribuição de proxy em massa",
|
||||
"testAll": "Testar todos",
|
||||
"errorTestFailed": "Falha ao testar proxies",
|
||||
"batchSelectedCount": "{count} selecionado(s)",
|
||||
"batchDeleteSelected": "Excluir {count} selecionado(s)",
|
||||
"batchActivateSelected": "Habilitar {count} selecionado(s)",
|
||||
"testPassed": "✓ OK",
|
||||
"close": "Fechar",
|
||||
"managePool": "Gerenciar pool",
|
||||
"poolTitle": "Pool e rotação de proxies",
|
||||
"poolDescription": "Anexe múltiplos proxies a um escopo e alterne os IPs de saída entre eles. Um escopo com um único proxy se comporta exatamente como uma atribuição simples.",
|
||||
"poolScopeIdLabel": "ID do escopo",
|
||||
"poolScopeIdPlaceholder": "id do provedor / id da conexão / id do combo",
|
||||
"poolScopeIdRequired": "Um ID de escopo é obrigatório para escopo de provedor, conta ou combo.",
|
||||
"poolLoad": "Carregar pool",
|
||||
"poolLoadFailed": "Falha ao carregar o pool de proxies",
|
||||
"poolStrategyLabel": "Estratégia de rotação",
|
||||
"poolStrategyHint": "round-robin percorre os membros em ordem; aleatório escolhe uniformemente; fixo mantém um membro por uma janela antes de avançar.",
|
||||
"poolStrategyFailed": "Falha ao atualizar a estratégia de rotação",
|
||||
"strategyRoundRobin": "Round-robin",
|
||||
"strategyRandom": "Aleatório",
|
||||
"strategySticky": "Fixo",
|
||||
"poolMembersLabel": "Membros do pool ({count})",
|
||||
"poolNoMembers": "Ainda não há proxies neste pool.",
|
||||
"poolRemove": "Remover",
|
||||
"poolRemoveFailed": "Falha ao remover o proxy do pool",
|
||||
"poolAddLabel": "Adicionar um proxy",
|
||||
"poolAddMember": "Adicionar",
|
||||
"poolAddFailed": "Falha ao adicionar o proxy ao pool",
|
||||
"poolSelectProxy": "Selecionar um proxy…",
|
||||
"strategyLatency": "Otimizado por latência"
|
||||
},
|
||||
"playground": {
|
||||
"title": "Title",
|
||||
@@ -7903,6 +8098,10 @@
|
||||
"topP": "Top-p",
|
||||
"presencePenalty": "Penalidade de presença",
|
||||
"frequencyPenalty": "Penalidade de frequência",
|
||||
"reasoningLabel": "Raciocínio",
|
||||
"thinking": "Pensando",
|
||||
"effort": "Esforço",
|
||||
"effortDefault": "Padrão",
|
||||
"seedPlaceholder": "Aleatório (deixar vazio)",
|
||||
"presetsLabel": "Presets",
|
||||
"loadPreset": "Carregar preset",
|
||||
@@ -8380,6 +8579,7 @@
|
||||
"saveMarketplaceUrl": "__MISSING__:Save & Reload",
|
||||
"marketplaceUrlSaved": "__MISSING__:Marketplace URL updated",
|
||||
"marketplaceEmpty": "__MISSING__:No plugins found in marketplace.",
|
||||
"marketplaceInstallComingSoon": "Instalações via marketplace em breve.",
|
||||
"verified": "__MISSING__:Verified",
|
||||
"install": "__MISSING__:Install",
|
||||
"installedFromMarketplace": "__MISSING__:Plugin {name} installed!",
|
||||
@@ -8512,6 +8712,7 @@
|
||||
"restartServer": "Reiniciar",
|
||||
"trustCert": "Confiar Cert",
|
||||
"downloadCert": "Baixar Cert",
|
||||
"certManualTitle": "O certificado não pôde ser instalado automaticamente (ex.: dentro de um contêiner). A ponte ainda pode ser executada — confie na CA manualmente:",
|
||||
"regenerateCert": "Regenerar Cert",
|
||||
"starting": "Iniciando…",
|
||||
"stopping": "Parando…",
|
||||
@@ -8938,6 +9139,68 @@
|
||||
"colScore": "__MISSING__:Score",
|
||||
"colAvgScore": "__MISSING__:Avg Score",
|
||||
"colModels": "__MISSING__:Models",
|
||||
"colType": "__MISSING__:Type"
|
||||
"colType": "__MISSING__:Type",
|
||||
"filterConfiguredOnly": "Somente configurados",
|
||||
"filterAvailableOnly": "Somente disponíveis",
|
||||
"filterAvailableOnlyHelp": "Ocultar provedores cujas conexões estão todas limitadas por taxa ou sem cota.",
|
||||
"configuredOnly": "Somente configurados",
|
||||
"configuredOnlyHint": "Mostrar apenas provedores com conexões ativas",
|
||||
"noConfiguredProviders": "Nenhum provedor configurado encontrado. Adicione uma conexão de provedor primeiro.",
|
||||
"colConfigured": "Status"
|
||||
},
|
||||
"discovery": {
|
||||
"title": "Descoberta de provedores",
|
||||
"subtitle": "Busque métodos de acesso gratuito/ilimitado em provedores e revise os achados. Opcional, somente local.",
|
||||
"scanLabel": "Provedor para buscar",
|
||||
"scanPlaceholder": "ex.: huggingchat",
|
||||
"scan": "Buscar",
|
||||
"scanning": "Buscando…",
|
||||
"scanQueued": "Busca concluída para {provider}.",
|
||||
"scanFailed": "Falha na busca.",
|
||||
"loadFailed": "Falha ao carregar os resultados da busca.",
|
||||
"localOnlyNote": "Esta ferramenta é somente local (loopback). As buscas são executadas nesta máquina e nunca são acessíveis remotamente.",
|
||||
"verify": "Verificar",
|
||||
"verifyFailed": "Falha ao verificar o achado.",
|
||||
"delete": "Excluir",
|
||||
"deleteFailed": "Falha ao excluir o achado.",
|
||||
"deleteTitle": "Excluir resultado de busca",
|
||||
"deleteConfirm": "Excluir o achado de busca para {provider}? Isso não pode ser desfeito.",
|
||||
"emptyTitle": "Ainda não há resultados de busca",
|
||||
"emptyDescription": "Execute uma busca acima para procurar métodos de acesso gratuito em um provedor.",
|
||||
"risk": "Risco",
|
||||
"method": "Método",
|
||||
"auth": "Autenticação",
|
||||
"feasibility": "Viabilidade",
|
||||
"models": "Modelos"
|
||||
},
|
||||
"chaosConfig": {
|
||||
"addProvider": "Adicionar Provedor",
|
||||
"configError": "Falha ao salvar a configuração do Chaos Mode",
|
||||
"configReset": "Restaurar Padrões",
|
||||
"configSaved": "Configuração do Chaos Mode salva com sucesso",
|
||||
"enableChaos": "Ativar Chaos Mode",
|
||||
"enableChaosDesc": "Permitir que chaves de API com o Chaos Mode ativado usem este recurso",
|
||||
"keyPermission": "Acesso ao Chaos Mode",
|
||||
"keyPermissionDesc": "Permitir que esta chave de API use o Chaos Mode (execução paralela multi-modelo)",
|
||||
"loadingProviderModels": "Carregando provedores...",
|
||||
"mode": "Modo Padrão",
|
||||
"modeCollaborative": "Colaborativo",
|
||||
"modeCollaborativeDesc": "Os modelos encadeiam saídas — cada um vê o resultado anterior",
|
||||
"modeParallel": "Paralelo",
|
||||
"modeParallelDesc": "Todos os modelos rodam simultaneamente — resultados mais rápidos",
|
||||
"modelId": "Modelo",
|
||||
"pageSubtitle": "Execute múltiplos modelos de IA em paralelo ou de forma colaborativa na mesma tarefa",
|
||||
"pageTitle": "Chaos Mode",
|
||||
"providerId": "Provedor",
|
||||
"providerOverrides": "Substituições de Provedor",
|
||||
"providerOverridesDesc": "Selecione modelos específicos por provedor para o Chaos Mode",
|
||||
"removeProvider": "Remover",
|
||||
"saveConfig": "Salvar Configuração",
|
||||
"systemPrompt": "Prompt de Sistema (opcional)",
|
||||
"systemPromptDesc": "Instruções personalizadas para todas as instâncias de modelo do Chaos Mode",
|
||||
"testButton": "Testar Chaos Mode",
|
||||
"testTask": "Escreva um poema curto sobre inteligência artificial",
|
||||
"timeout": "Tempo Limite (ms)",
|
||||
"timeoutDesc": "Tempo máximo por chamada de modelo (5000-600000ms)"
|
||||
}
|
||||
}
|
||||
|
||||
203
src/lib/db/interceptionRules.ts
Normal file
203
src/lib/db/interceptionRules.ts
Normal file
@@ -0,0 +1,203 @@
|
||||
/**
|
||||
* db/interceptionRules.ts — Per-model web-search / web-fetch interception rules (#3384).
|
||||
*
|
||||
* CRUD against the key_value table under namespace "interception_rules". Follows the
|
||||
* established key_value pattern from paramFilters.ts / databaseSettings.ts.
|
||||
*
|
||||
* Resolution precedence (see resolveInterceptSearch): per-model rule > provider-level
|
||||
* rule > undefined (caller falls back to the existing native-bypass defaults).
|
||||
*/
|
||||
|
||||
import { getDbInstance } from "./core";
|
||||
|
||||
const NAMESPACE = "interception_rules";
|
||||
|
||||
// ── Types ───────────────────────────────────────────────────────────────────
|
||||
|
||||
export type FetchInterceptionBackend = "firecrawl" | "jina" | "tavily";
|
||||
|
||||
export interface ModelInterceptionRule {
|
||||
/** true = route through OmniRoute's /v1/search; false = force native passthrough. */
|
||||
interceptSearch?: boolean;
|
||||
/** true = route through OmniRoute's /v1/web/fetch; false = force native passthrough. */
|
||||
interceptFetch?: boolean;
|
||||
fetchBackend?: FetchInterceptionBackend;
|
||||
fetchProxyUrl?: string;
|
||||
}
|
||||
|
||||
export interface ProviderInterceptionRules {
|
||||
/** Provider-level default, used when a model has no override. */
|
||||
interceptSearch?: boolean;
|
||||
interceptFetch?: boolean;
|
||||
fetchBackend?: FetchInterceptionBackend;
|
||||
fetchProxyUrl?: string;
|
||||
/** Per-model overrides (stricter/looser than provider-level). */
|
||||
models?: Record<string, ModelInterceptionRule>;
|
||||
}
|
||||
|
||||
// ── Cache ───────────────────────────────────────────────────────────────────
|
||||
|
||||
let rulesCache: Map<string, ProviderInterceptionRules> | null = null;
|
||||
|
||||
function invalidateCache(): void {
|
||||
rulesCache = null;
|
||||
}
|
||||
|
||||
// ── Helpers ─────────────────────────────────────────────────────────────────
|
||||
|
||||
function isRecord(value: unknown): value is Record<string, unknown> {
|
||||
return value !== null && typeof value === "object" && !Array.isArray(value);
|
||||
}
|
||||
|
||||
function toNormalizedString(value: unknown): string | null {
|
||||
return typeof value === "string" && value.trim().length > 0 ? value.trim() : null;
|
||||
}
|
||||
|
||||
function toOptionalBool(value: unknown): boolean | undefined {
|
||||
return typeof value === "boolean" ? value : undefined;
|
||||
}
|
||||
|
||||
function toFetchBackend(value: unknown): FetchInterceptionBackend | undefined {
|
||||
return value === "firecrawl" || value === "jina" || value === "tavily" ? value : undefined;
|
||||
}
|
||||
|
||||
function parseStoredValue(raw: unknown): unknown {
|
||||
if (typeof raw !== "string") return raw;
|
||||
try {
|
||||
return JSON.parse(raw);
|
||||
} catch {
|
||||
return raw;
|
||||
}
|
||||
}
|
||||
|
||||
function toModelInterceptionRule(raw: unknown): ModelInterceptionRule | null {
|
||||
if (!isRecord(raw)) return null;
|
||||
const rule: ModelInterceptionRule = {
|
||||
interceptSearch: toOptionalBool(raw.interceptSearch),
|
||||
interceptFetch: toOptionalBool(raw.interceptFetch),
|
||||
fetchBackend: toFetchBackend(raw.fetchBackend),
|
||||
fetchProxyUrl: toNormalizedString(raw.fetchProxyUrl) ?? undefined,
|
||||
};
|
||||
const hasAnyField = Object.values(rule).some((v) => v !== undefined);
|
||||
return hasAnyField ? rule : null;
|
||||
}
|
||||
|
||||
function toModelInterceptionRules(raw: unknown): Record<string, ModelInterceptionRule> {
|
||||
const models: Record<string, ModelInterceptionRule> = {};
|
||||
if (!isRecord(raw)) return models;
|
||||
for (const [modelId, val] of Object.entries(raw)) {
|
||||
const rule = toModelInterceptionRule(val);
|
||||
if (rule) models[modelId] = rule;
|
||||
}
|
||||
return models;
|
||||
}
|
||||
|
||||
function toProviderInterceptionRules(raw: unknown): ProviderInterceptionRules | null {
|
||||
if (!isRecord(raw)) return null;
|
||||
const models = toModelInterceptionRules(raw.models);
|
||||
return {
|
||||
interceptSearch: toOptionalBool(raw.interceptSearch),
|
||||
interceptFetch: toOptionalBool(raw.interceptFetch),
|
||||
fetchBackend: toFetchBackend(raw.fetchBackend),
|
||||
fetchProxyUrl: toNormalizedString(raw.fetchProxyUrl) ?? undefined,
|
||||
models: Object.keys(models).length > 0 ? models : undefined,
|
||||
};
|
||||
}
|
||||
|
||||
// ── Read ────────────────────────────────────────────────────────────────────
|
||||
|
||||
function readNamespace(namespace: string): Record<string, unknown> {
|
||||
const db = getDbInstance();
|
||||
const rows = db
|
||||
.prepare("SELECT key, value FROM key_value WHERE namespace = ?")
|
||||
.all(namespace) as Array<{ key: string; value: string }>;
|
||||
|
||||
const values: Record<string, unknown> = {};
|
||||
for (const row of rows) {
|
||||
values[row.key] = parseStoredValue(row.value);
|
||||
}
|
||||
return values;
|
||||
}
|
||||
|
||||
function loadAllRules(): Map<string, ProviderInterceptionRules> {
|
||||
const raw = readNamespace(NAMESPACE);
|
||||
const map = new Map<string, ProviderInterceptionRules>();
|
||||
for (const [key, value] of Object.entries(raw)) {
|
||||
const parsed = toProviderInterceptionRules(value);
|
||||
if (parsed) map.set(key, parsed);
|
||||
}
|
||||
return map;
|
||||
}
|
||||
|
||||
function loadRulesCached(): Map<string, ProviderInterceptionRules> {
|
||||
if (rulesCache === null) {
|
||||
rulesCache = loadAllRules();
|
||||
}
|
||||
return rulesCache;
|
||||
}
|
||||
|
||||
// ── Public API ──────────────────────────────────────────────────────────────
|
||||
|
||||
/** Get the interception rules for a single provider, or null if not configured. */
|
||||
export function getInterceptionRules(provider: string): ProviderInterceptionRules | null {
|
||||
return toNormalizedString(provider) ? (loadRulesCached().get(provider) ?? null) : null;
|
||||
}
|
||||
|
||||
/** Upsert the entire interception rule set for a provider. Invalidates the cache. */
|
||||
export function setInterceptionRules(provider: string, rules: ProviderInterceptionRules): void {
|
||||
const normalizedProvider = toNormalizedString(provider);
|
||||
if (!normalizedProvider) return;
|
||||
|
||||
const db = getDbInstance();
|
||||
const stmt = db.prepare(
|
||||
"INSERT OR REPLACE INTO key_value (namespace, key, value) VALUES (?, ?, ?)"
|
||||
);
|
||||
|
||||
const normalized: ProviderInterceptionRules = {
|
||||
interceptSearch: rules.interceptSearch,
|
||||
interceptFetch: rules.interceptFetch,
|
||||
fetchBackend: rules.fetchBackend,
|
||||
fetchProxyUrl: rules.fetchProxyUrl,
|
||||
models: rules.models && Object.keys(rules.models).length > 0 ? rules.models : undefined,
|
||||
};
|
||||
|
||||
stmt.run(NAMESPACE, normalizedProvider, JSON.stringify(normalized));
|
||||
invalidateCache();
|
||||
}
|
||||
|
||||
/** Delete the interception rules for a provider. Resets that provider to default behavior. */
|
||||
export function deleteInterceptionRules(provider: string): void {
|
||||
const normalizedProvider = toNormalizedString(provider);
|
||||
if (!normalizedProvider) return;
|
||||
|
||||
const db = getDbInstance();
|
||||
db.prepare("DELETE FROM key_value WHERE namespace = ? AND key = ?").run(
|
||||
NAMESPACE,
|
||||
normalizedProvider
|
||||
);
|
||||
invalidateCache();
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the effective `interceptSearch` override for a provider/model pair.
|
||||
*
|
||||
* Precedence: per-model rule > provider-level rule > undefined (no override — the
|
||||
* caller should fall back to the existing native-bypass defaults).
|
||||
*/
|
||||
export function resolveInterceptSearch(
|
||||
provider: string | null | undefined,
|
||||
model: string | null | undefined
|
||||
): boolean | undefined {
|
||||
const normalizedProvider = toNormalizedString(provider);
|
||||
if (!normalizedProvider) return undefined;
|
||||
|
||||
const rules = getInterceptionRules(normalizedProvider);
|
||||
if (!rules) return undefined;
|
||||
|
||||
const normalizedModel = toNormalizedString(model);
|
||||
if (normalizedModel && rules.models?.[normalizedModel]?.interceptSearch !== undefined) {
|
||||
return rules.models[normalizedModel].interceptSearch;
|
||||
}
|
||||
|
||||
return rules.interceptSearch;
|
||||
}
|
||||
16
src/lib/db/migrations/120_interception_rules.sql
Normal file
16
src/lib/db/migrations/120_interception_rules.sql
Normal file
@@ -0,0 +1,16 @@
|
||||
-- 120_interception_rules.sql
|
||||
-- Documents the interception_rules namespace in the key_value table (#3384).
|
||||
-- No schema change — the key_value table already exists.
|
||||
-- Key = provider ID, value = JSON with shape:
|
||||
-- {
|
||||
-- interceptSearch?: boolean,
|
||||
-- interceptFetch?: boolean,
|
||||
-- fetchBackend?: "firecrawl"|"jina"|"tavily",
|
||||
-- fetchProxyUrl?: string,
|
||||
-- models?: { [modelId]: { interceptSearch?, interceptFetch?, fetchBackend?, fetchProxyUrl? } }
|
||||
-- }
|
||||
--
|
||||
-- Resolution precedence: per-model rule > provider-level rule > undefined (caller
|
||||
-- falls back to the existing native web-search-bypass defaults in webSearchFallback.ts).
|
||||
--
|
||||
-- See: src/lib/db/interceptionRules.ts
|
||||
@@ -788,7 +788,6 @@ export type {
|
||||
// proxy_logs — export query (#3500 slice 4)
|
||||
// ---------------------------------------------------------------------------
|
||||
export { exportProxyLogsSince } from "./db/proxyLogs";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Per-connection 429 cooldown wrappers (#5957 / #5958 — Issue 1 follow-ups)
|
||||
// Logic lives in db/providers/rateLimit.ts (Hard Rule #2 — localDb is re-export
|
||||
@@ -797,3 +796,4 @@ export { exportProxyLogsSince } from "./db/proxyLogs";
|
||||
export { markConnectionRateLimitedUntil, clearConnectionRateLimit } from "./db/providers";
|
||||
// Provider param filters — denylist/allowlist config per provider/model (#6625)
|
||||
export * from "./db/paramFilters";
|
||||
export * from "./db/interceptionRules"; // Per-model web-search/web-fetch interception rules (#3384)
|
||||
|
||||
113
src/shared/components/Sidebar.search.test.tsx
Normal file
113
src/shared/components/Sidebar.search.test.tsx
Normal file
@@ -0,0 +1,113 @@
|
||||
// @vitest-environment jsdom
|
||||
import React from "react";
|
||||
import { act } from "react";
|
||||
import { createRoot, type Root } from "react-dom/client";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
// Skip CloudSyncStatus entirely (it polls /api/sync/cloud + uses next/navigation's
|
||||
// useRouter, which we don't otherwise need to mock for this component).
|
||||
process.env.NEXT_PUBLIC_OMNIROUTE_E2E_MODE = "1";
|
||||
|
||||
vi.mock("next-intl", () => ({
|
||||
useTranslations: () => {
|
||||
const translate = (key: string) => key;
|
||||
translate.has = () => false;
|
||||
return translate;
|
||||
},
|
||||
}));
|
||||
|
||||
vi.mock("next/navigation", () => ({
|
||||
usePathname: () => "/dashboard/combos",
|
||||
}));
|
||||
|
||||
const cleanupCallbacks: Array<() => void> = [];
|
||||
|
||||
function makeContainer(): HTMLElement {
|
||||
const container = document.createElement("div");
|
||||
document.body.appendChild(container);
|
||||
cleanupCallbacks.push(() => container.remove());
|
||||
return container;
|
||||
}
|
||||
|
||||
function jsonResponse(body: unknown) {
|
||||
return { ok: true, status: 200, json: async () => body } as Response;
|
||||
}
|
||||
|
||||
describe("Sidebar search/filter (#4013)", () => {
|
||||
let root: Root | undefined;
|
||||
|
||||
beforeEach(() => {
|
||||
(
|
||||
globalThis as typeof globalThis & { IS_REACT_ACT_ENVIRONMENT?: boolean }
|
||||
).IS_REACT_ACT_ENVIRONMENT = true;
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
vi.fn(async (url: string) => {
|
||||
if (String(url).includes("/api/settings")) return jsonResponse({});
|
||||
return jsonResponse({});
|
||||
})
|
||||
);
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
if (root) {
|
||||
act(() => root!.unmount());
|
||||
root = undefined;
|
||||
}
|
||||
while (cleanupCallbacks.length > 0) cleanupCallbacks.pop()?.();
|
||||
document.body.innerHTML = "";
|
||||
vi.unstubAllGlobals();
|
||||
vi.resetModules();
|
||||
});
|
||||
|
||||
it("renders a search input at the top of the expanded sidebar", async () => {
|
||||
// First import in this file pays the one-time cost of compiling Sidebar's
|
||||
// large transitive dependency graph (sidebarVisibility sections, icons, etc).
|
||||
const { default: Sidebar } = await import("@/shared/components/Sidebar");
|
||||
const container = makeContainer();
|
||||
root = createRoot(container);
|
||||
await act(async () => {
|
||||
root!.render(<Sidebar />);
|
||||
});
|
||||
|
||||
const input = container.querySelector('input[type="search"]');
|
||||
expect(input).toBeTruthy();
|
||||
}, 20000);
|
||||
|
||||
it("filters visible nav items down to those matching the typed query", async () => {
|
||||
const { default: Sidebar } = await import("@/shared/components/Sidebar");
|
||||
const container = makeContainer();
|
||||
root = createRoot(container);
|
||||
await act(async () => {
|
||||
root!.render(<Sidebar />);
|
||||
});
|
||||
|
||||
const linksBefore = container.querySelectorAll("nav a");
|
||||
expect(linksBefore.length).toBeGreaterThan(1);
|
||||
|
||||
const input = container.querySelector('input[type="search"]') as HTMLInputElement;
|
||||
expect(input).toBeTruthy();
|
||||
|
||||
const nativeSetter = Object.getOwnPropertyDescriptor(
|
||||
HTMLInputElement.prototype,
|
||||
"value"
|
||||
)!.set!;
|
||||
|
||||
await act(async () => {
|
||||
nativeSetter.call(input, "zzz-no-such-nav-item-zzz");
|
||||
input.dispatchEvent(new Event("input", { bubbles: true }));
|
||||
});
|
||||
|
||||
const linksAfterNoMatch = container.querySelectorAll("nav a");
|
||||
expect(linksAfterNoMatch.length).toBe(0);
|
||||
expect(container.querySelector("nav")?.textContent).toBeTruthy();
|
||||
|
||||
await act(async () => {
|
||||
nativeSetter.call(input, "");
|
||||
input.dispatchEvent(new Event("input", { bubbles: true }));
|
||||
});
|
||||
|
||||
const linksAfterClear = container.querySelectorAll("nav a");
|
||||
expect(linksAfterClear.length).toBe(linksBefore.length);
|
||||
});
|
||||
});
|
||||
@@ -5,9 +5,11 @@ import Link from "next/link";
|
||||
import { usePathname } from "next/navigation";
|
||||
import { cn } from "@/shared/utils/cn";
|
||||
import { getActiveSidebarHref } from "@/shared/utils/sidebarRouteMatch";
|
||||
import { filterSidebarSectionsByQuery } from "@/shared/utils/sidebarSearch";
|
||||
import { APP_CONFIG } from "@/shared/constants/appConfig";
|
||||
import OmniRouteLogo from "./OmniRouteLogo";
|
||||
import Button from "./Button";
|
||||
import Input from "./Input";
|
||||
import { ConfirmModal } from "./Modal";
|
||||
import CloudSyncStatus from "./CloudSyncStatus";
|
||||
import { useTranslations } from "next-intl";
|
||||
@@ -101,6 +103,7 @@ export default function Sidebar({
|
||||
);
|
||||
const [pinnedSections, setPinnedSections] = useState<Set<SidebarSectionId>>(new Set());
|
||||
const [hoveredItem, setHoveredItem] = useState<HoveredItem>(null);
|
||||
const [searchQuery, setSearchQuery] = useState("");
|
||||
|
||||
// Load persisted state on mount; OmniProxy is pinned by default on first visit
|
||||
useEffect(() => {
|
||||
@@ -266,6 +269,11 @@ export default function Sidebar({
|
||||
|
||||
const activeHref = getActiveSidebarHref(pathname, allVisibleItems);
|
||||
|
||||
const isSearching = searchQuery.trim().length > 0;
|
||||
const displaySections = isSearching
|
||||
? filterSidebarSectionsByQuery(visibleSections, searchQuery)
|
||||
: visibleSections;
|
||||
|
||||
// Auto-expand the section containing the active page (without closing others)
|
||||
useEffect(() => {
|
||||
if (collapsed) return;
|
||||
@@ -518,6 +526,21 @@ export default function Sidebar({
|
||||
</Link>
|
||||
</div>
|
||||
|
||||
{!collapsed && (
|
||||
<div className="px-4 pb-2">
|
||||
<Input
|
||||
type="search"
|
||||
value={searchQuery}
|
||||
onChange={(e) => setSearchQuery(e.target.value)}
|
||||
placeholder={tc("search")}
|
||||
aria-label={tc("search")}
|
||||
icon="search"
|
||||
className="gap-0"
|
||||
inputClassName="py-1.5 text-xs"
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
|
||||
<nav
|
||||
aria-label="Main navigation"
|
||||
className={cn(
|
||||
@@ -525,9 +548,12 @@ export default function Sidebar({
|
||||
collapsed ? "px-2 space-y-0.5" : "px-3"
|
||||
)}
|
||||
>
|
||||
{visibleSections.map((section, idx) => {
|
||||
{isSearching && displaySections.length === 0 && (
|
||||
<p className="px-2 py-3 text-xs text-text-muted/60">{tc("noResults")}</p>
|
||||
)}
|
||||
{displaySections.map((section, idx) => {
|
||||
const sectionId = section.id as SidebarSectionId;
|
||||
const isExpanded = expandedSections.has(sectionId);
|
||||
const isExpanded = isSearching || expandedSections.has(sectionId);
|
||||
const isPinned = pinnedSections.has(sectionId);
|
||||
const isFirst = idx === 0;
|
||||
const sectionItems = section.children.flatMap((child: any) =>
|
||||
|
||||
68
src/shared/utils/sidebarSearch.ts
Normal file
68
src/shared/utils/sidebarSearch.ts
Normal file
@@ -0,0 +1,68 @@
|
||||
// Pure, framework-free filtering for the sidebar quick-search (#4013).
|
||||
// Operates on the already-resolved (labeled) section/child shape produced by
|
||||
// Sidebar.tsx, so it has no dependency on next-intl/React and stays trivially
|
||||
// unit-testable.
|
||||
|
||||
export interface SearchableLabeled {
|
||||
label: string;
|
||||
}
|
||||
|
||||
export interface SearchableGroup<TItem extends SearchableLabeled> {
|
||||
type: "group";
|
||||
items: readonly TItem[];
|
||||
}
|
||||
|
||||
export type SearchableChild<TItem extends SearchableLabeled> =
|
||||
| TItem
|
||||
| SearchableGroup<TItem>;
|
||||
|
||||
export interface SearchableSection<TItem extends SearchableLabeled> {
|
||||
children: readonly SearchableChild<TItem>[];
|
||||
}
|
||||
|
||||
function isGroupChild<TItem extends SearchableLabeled>(
|
||||
child: SearchableChild<TItem>
|
||||
): child is SearchableGroup<TItem> {
|
||||
return (
|
||||
typeof child === "object" &&
|
||||
child !== null &&
|
||||
"type" in child &&
|
||||
(child as { type?: unknown }).type === "group"
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Filters sidebar sections by a free-text query matched (case-insensitive,
|
||||
* substring) against each item's resolved label. Groups are kept only if at
|
||||
* least one of their items still matches; sections are kept only if at least
|
||||
* one child (flat item or non-empty group) still matches. Passing an empty/
|
||||
* whitespace-only query returns the input sections unchanged.
|
||||
*/
|
||||
export function filterSidebarSectionsByQuery<
|
||||
TItem extends SearchableLabeled,
|
||||
TSection extends SearchableSection<TItem>,
|
||||
>(sections: readonly TSection[], query: string): TSection[] {
|
||||
const needle = query.trim().toLowerCase();
|
||||
if (!needle) return [...sections];
|
||||
|
||||
const matches = (item: TItem) => item.label.toLowerCase().includes(needle);
|
||||
|
||||
const result: TSection[] = [];
|
||||
for (const section of sections) {
|
||||
const children: SearchableChild<TItem>[] = [];
|
||||
for (const child of section.children) {
|
||||
if (isGroupChild(child)) {
|
||||
const items = child.items.filter(matches);
|
||||
if (items.length > 0) {
|
||||
children.push({ ...child, items });
|
||||
}
|
||||
} else if (matches(child)) {
|
||||
children.push(child);
|
||||
}
|
||||
}
|
||||
if (children.length > 0) {
|
||||
result.push({ ...section, children } as TSection);
|
||||
}
|
||||
}
|
||||
return result;
|
||||
}
|
||||
@@ -3,6 +3,20 @@ import assert from "node:assert";
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
|
||||
function flatten(obj: Record<string, unknown>, prefix = ""): Record<string, unknown> {
|
||||
const out: Record<string, unknown> = {};
|
||||
for (const k of Object.keys(obj)) {
|
||||
const key = prefix ? `${prefix}.${k}` : k;
|
||||
const v = obj[k];
|
||||
if (v && typeof v === "object" && !Array.isArray(v)) {
|
||||
Object.assign(out, flatten(v as Record<string, unknown>, key));
|
||||
} else {
|
||||
out[key] = v;
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
describe("i18n pt-BR integrity", () => {
|
||||
it("should be a valid JSON file", () => {
|
||||
const ptPath = path.resolve("src/i18n/messages/pt-BR.json");
|
||||
@@ -23,4 +37,29 @@ describe("i18n pt-BR integrity", () => {
|
||||
assert.ok(json.cache.loadingCacheAria);
|
||||
assert.ok(json.analytics.usageAnalyticsTitle);
|
||||
});
|
||||
|
||||
// Regression guard for #6695: en.json gained 194 keys that were never
|
||||
// mirrored into pt-BR.json (i18n:sync-ui was not re-run), and the
|
||||
// i18n-ui-coverage CI gate is a percentage threshold (80%) so it stayed
|
||||
// green at 93.8% coverage despite the gap. This asserts full key parity
|
||||
// so a future drift fails loudly instead of silently degrading coverage.
|
||||
it("should contain every key present in en.json (no drift, #6695)", () => {
|
||||
const enPath = path.resolve("src/i18n/messages/en.json");
|
||||
const ptPath = path.resolve("src/i18n/messages/pt-BR.json");
|
||||
const en = JSON.parse(fs.readFileSync(enPath, "utf8"));
|
||||
const pt = JSON.parse(fs.readFileSync(ptPath, "utf8"));
|
||||
|
||||
const enFlat = flatten(en);
|
||||
const ptFlat = flatten(pt);
|
||||
|
||||
const missing = Object.keys(enFlat).filter((k) => !(k in ptFlat));
|
||||
|
||||
assert.strictEqual(
|
||||
missing.length,
|
||||
0,
|
||||
`pt-BR.json is missing ${missing.length} keys present in en.json. Sample: ${missing
|
||||
.slice(0, 10)
|
||||
.join(", ")}`
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
96
tests/unit/interception-rules.test.ts
Normal file
96
tests/unit/interception-rules.test.ts
Normal file
@@ -0,0 +1,96 @@
|
||||
import { describe, it, beforeEach, after } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import fs from "node:fs";
|
||||
|
||||
// Set DATA_DIR to a temp dir before any imports that touch the DB.
|
||||
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-test-interception-rules-"));
|
||||
process.env.DATA_DIR = tmpDir;
|
||||
|
||||
const core = await import("../../src/lib/db/core.ts");
|
||||
const {
|
||||
getInterceptionRules,
|
||||
setInterceptionRules,
|
||||
deleteInterceptionRules,
|
||||
resolveInterceptSearch,
|
||||
} = await import("../../src/lib/db/interceptionRules.ts");
|
||||
|
||||
// #3384 — per-model web-search/web-fetch interception rule store.
|
||||
describe("db/interceptionRules — per-model interception rules (#3384)", () => {
|
||||
function resetDb() {
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||
fs.mkdirSync(tmpDir, { recursive: true });
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
resetDb();
|
||||
});
|
||||
|
||||
after(() => {
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it("returns null for an unconfigured provider", () => {
|
||||
assert.equal(getInterceptionRules("anthropic"), null);
|
||||
});
|
||||
|
||||
it("round-trips a provider-level rule via set/get", () => {
|
||||
setInterceptionRules("anthropic", { interceptSearch: true, interceptFetch: false });
|
||||
const rules = getInterceptionRules("anthropic");
|
||||
assert.equal(rules?.interceptSearch, true);
|
||||
assert.equal(rules?.interceptFetch, false);
|
||||
});
|
||||
|
||||
it("round-trips a per-model override", () => {
|
||||
setInterceptionRules("anthropic", {
|
||||
interceptSearch: false,
|
||||
models: { "claude-opus-4": { interceptSearch: true } },
|
||||
});
|
||||
const rules = getInterceptionRules("anthropic");
|
||||
assert.equal(rules?.interceptSearch, false);
|
||||
assert.equal(rules?.models?.["claude-opus-4"]?.interceptSearch, true);
|
||||
});
|
||||
|
||||
it("delete resets a provider back to unconfigured", () => {
|
||||
setInterceptionRules("anthropic", { interceptSearch: true });
|
||||
deleteInterceptionRules("anthropic");
|
||||
assert.equal(getInterceptionRules("anthropic"), null);
|
||||
});
|
||||
|
||||
it("invalidates the in-memory cache after a write", () => {
|
||||
setInterceptionRules("openai", { interceptSearch: true });
|
||||
assert.equal(getInterceptionRules("openai")?.interceptSearch, true);
|
||||
setInterceptionRules("openai", { interceptSearch: false });
|
||||
assert.equal(getInterceptionRules("openai")?.interceptSearch, false);
|
||||
});
|
||||
|
||||
describe("resolveInterceptSearch — precedence", () => {
|
||||
it("returns undefined when no rule is configured (caller falls back to bypass defaults)", () => {
|
||||
assert.equal(resolveInterceptSearch("anthropic", "claude-opus-4"), undefined);
|
||||
});
|
||||
|
||||
it("returns the provider-level rule when no model override exists", () => {
|
||||
setInterceptionRules("anthropic", { interceptSearch: true });
|
||||
assert.equal(resolveInterceptSearch("anthropic", "claude-opus-4"), true);
|
||||
assert.equal(resolveInterceptSearch("anthropic", "claude-haiku-4"), true);
|
||||
});
|
||||
|
||||
it("per-model rule overrides the provider-level rule", () => {
|
||||
setInterceptionRules("anthropic", {
|
||||
interceptSearch: false,
|
||||
models: { "claude-opus-4": { interceptSearch: true } },
|
||||
});
|
||||
assert.equal(resolveInterceptSearch("anthropic", "claude-opus-4"), true);
|
||||
assert.equal(resolveInterceptSearch("anthropic", "claude-haiku-4"), false);
|
||||
});
|
||||
|
||||
it("returns undefined for an empty/missing provider", () => {
|
||||
assert.equal(resolveInterceptSearch("", "claude-opus-4"), undefined);
|
||||
assert.equal(resolveInterceptSearch(null, "claude-opus-4"), undefined);
|
||||
assert.equal(resolveInterceptSearch(undefined, "claude-opus-4"), undefined);
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -73,6 +73,13 @@ const EXPECTED_PROVIDER_KEYS = [
|
||||
"zed-hosted",
|
||||
];
|
||||
|
||||
const browserUrl = "http://localhost:20128/callback";
|
||||
const publicBaseEnv = {
|
||||
NEXT_PUBLIC_BASE_URL: "https://omniroute.example.com",
|
||||
ANTIGRAVITY_OAUTH_CLIENT_ID: "custom-antigravity.apps.googleusercontent.com",
|
||||
ANTIGRAVITY_OAUTH_CLIENT_SECRET: "custom-antigravity-secret",
|
||||
};
|
||||
|
||||
const EXPECTED_CONFIG_BY_PROVIDER = {
|
||||
claude: CLAUDE_CONFIG,
|
||||
codex: CODEX_CONFIG,
|
||||
@@ -98,6 +105,17 @@ const EXPECTED_CONFIG_BY_PROVIDER = {
|
||||
"zed-hosted": ZED_HOSTED_CONFIG,
|
||||
};
|
||||
|
||||
const KIRO_REQUIRED_FIELDS = [
|
||||
"registerClientUrl",
|
||||
"deviceAuthUrl",
|
||||
"tokenUrl",
|
||||
"socialAuthEndpoint",
|
||||
"socialLoginUrl",
|
||||
"socialTokenUrl",
|
||||
"socialRefreshUrl",
|
||||
"authMethods",
|
||||
];
|
||||
|
||||
const REQUIRED_FIELDS_BY_PROVIDER = {
|
||||
claude: ["authorizeUrl", "tokenUrl", "redirectUri", "scopes", "clientId"],
|
||||
codex: ["authorizeUrl", "tokenUrl", "scope", "clientId"],
|
||||
@@ -117,26 +135,8 @@ const REQUIRED_FIELDS_BY_PROVIDER = {
|
||||
"codeChallengeMethod",
|
||||
"clientId",
|
||||
],
|
||||
kiro: [
|
||||
"registerClientUrl",
|
||||
"deviceAuthUrl",
|
||||
"tokenUrl",
|
||||
"socialAuthEndpoint",
|
||||
"socialLoginUrl",
|
||||
"socialTokenUrl",
|
||||
"socialRefreshUrl",
|
||||
"authMethods",
|
||||
],
|
||||
"amazon-q": [
|
||||
"registerClientUrl",
|
||||
"deviceAuthUrl",
|
||||
"tokenUrl",
|
||||
"socialAuthEndpoint",
|
||||
"socialLoginUrl",
|
||||
"socialTokenUrl",
|
||||
"socialRefreshUrl",
|
||||
"authMethods",
|
||||
],
|
||||
kiro: KIRO_REQUIRED_FIELDS,
|
||||
"amazon-q": KIRO_REQUIRED_FIELDS,
|
||||
cursor: ["apiEndpoint", "api3Endpoint", "agentEndpoint", "agentNonPrivacyEndpoint", "dbKeys"],
|
||||
kilocode: ["apiBaseUrl", "initiateUrl", "pollUrlBase"],
|
||||
cline: ["appBaseUrl", "apiBaseUrl", "authorizeUrl", "tokenExchangeUrl", "refreshUrl"],
|
||||
@@ -339,10 +339,6 @@ test("browser-based providers expose buildAuthUrl and return provider-specific a
|
||||
assert.equal(clineUrl.origin, "https://api.cline.bot");
|
||||
});
|
||||
|
||||
// zed-hosted's buildAuthUrl deliberately returns an object (authUrl + codeVerifier +
|
||||
// redirectUri) instead of a bare string — generateAuthData() in providers.ts special-
|
||||
// cases this shape to thread an RSA private-key verifier through the existing PKCE
|
||||
// codeVerifier slot (see src/lib/oauth/providers/zed-hosted.ts header comment).
|
||||
test("zed-hosted buildAuthUrl returns {authUrl, codeVerifier, redirectUri} carrying a fresh RSA keypair", () => {
|
||||
const built = PROVIDERS["zed-hosted"].buildAuthUrl(ZED_HOSTED_CONFIG);
|
||||
assert.equal(typeof built, "object");
|
||||
@@ -361,8 +357,6 @@ test("generateAuthData honors an object-returning buildAuthUrl (zed-hosted) with
|
||||
assert.ok(zedAuthData.codeVerifier.startsWith("zed-rsa-pkcs1:"));
|
||||
assert.ok(zedAuthData.redirectUri.startsWith("http://127.0.0.1:"));
|
||||
|
||||
// A string-returning provider (cline) must still get the plain PKCE codeVerifier,
|
||||
// not be affected by the object-return branch added for zed-hosted.
|
||||
const clineAuthData = oauthHelpers.generateAuthData("cline", "http://localhost:20128/callback");
|
||||
assert.equal(typeof clineAuthData.authUrl, "string");
|
||||
assert.equal(clineAuthData.redirectUri, "http://localhost:20128/callback");
|
||||
@@ -370,15 +364,10 @@ test("generateAuthData honors an object-returning buildAuthUrl (zed-hosted) with
|
||||
assert.ok(!clineAuthData.codeVerifier.startsWith("zed-rsa-pkcs1:"));
|
||||
});
|
||||
|
||||
// Regression for #3861: GitLab Duo needs an operator-registered OAuth client_id.
|
||||
// When it's missing, buildAuthUrl must return null (like Qoder) so the authorize route
|
||||
// can surface a clear "configure it" message — it previously THREW, which the route
|
||||
// swallowed into an opaque "Internal server error" 500 at the Add Connection step.
|
||||
test("gitlab-duo buildAuthUrl returns null (not throw) when client_id is unconfigured (#3861)", () => {
|
||||
const redirectUri = "http://localhost:20128/callback";
|
||||
const unconfigured = PROVIDERS["gitlab-duo"].buildAuthUrl(
|
||||
{ ...GITLAB_DUO_CONFIG, clientId: "" },
|
||||
redirectUri,
|
||||
browserUrl,
|
||||
"state-x",
|
||||
"challenge-y"
|
||||
);
|
||||
@@ -386,22 +375,14 @@ test("gitlab-duo buildAuthUrl returns null (not throw) when client_id is unconfi
|
||||
|
||||
// Configured: returns a real authorize URL carrying the client_id + PKCE challenge.
|
||||
const configured = new URL(
|
||||
PROVIDERS["gitlab-duo"].buildAuthUrl(GITLAB_DUO_CONFIG, redirectUri, "state-x", "challenge-y")
|
||||
PROVIDERS["gitlab-duo"].buildAuthUrl(GITLAB_DUO_CONFIG, browserUrl, "state-x", "challenge-y")
|
||||
);
|
||||
assert.equal(configured.searchParams.get("client_id"), GITLAB_DUO_CONFIG.clientId);
|
||||
assert.equal(configured.searchParams.get("code_challenge"), "challenge-y");
|
||||
});
|
||||
|
||||
test("custom Google OAuth credentials switch Antigravity remote callbacks to NEXT_PUBLIC_BASE_URL", () => {
|
||||
const redirectUri = resolveBrowserOAuthRedirectUri(
|
||||
"antigravity",
|
||||
"http://localhost:20128/callback",
|
||||
{
|
||||
NEXT_PUBLIC_BASE_URL: "https://omniroute.example.com/",
|
||||
ANTIGRAVITY_OAUTH_CLIENT_ID: "custom-antigravity.apps.googleusercontent.com",
|
||||
ANTIGRAVITY_OAUTH_CLIENT_SECRET: "custom-antigravity-secret",
|
||||
}
|
||||
);
|
||||
const redirectUri = resolveBrowserOAuthRedirectUri("antigravity", browserUrl, publicBaseEnv);
|
||||
|
||||
assert.equal(redirectUri, "https://omniroute.example.com/callback");
|
||||
});
|
||||
@@ -410,32 +391,28 @@ test("custom Google OAuth callbacks preserve the requested callback path and que
|
||||
const redirectUri = resolveBrowserOAuthRedirectUri(
|
||||
"antigravity",
|
||||
"http://127.0.0.1:20128/auth/callback?source=popup",
|
||||
{
|
||||
NEXT_PUBLIC_BASE_URL: "https://omniroute.example.com/base",
|
||||
ANTIGRAVITY_OAUTH_CLIENT_ID: "custom-antigravity.apps.googleusercontent.com",
|
||||
ANTIGRAVITY_OAUTH_CLIENT_SECRET: "custom-antigravity-secret",
|
||||
}
|
||||
{ ...publicBaseEnv, NEXT_PUBLIC_BASE_URL: "https://omniroute.example.com/base" }
|
||||
);
|
||||
|
||||
assert.equal(redirectUri, "https://omniroute.example.com/base/auth/callback?source=popup");
|
||||
});
|
||||
|
||||
test("custom Google OAuth credentials switch IPv6 loopback callbacks to public base URL", () => {
|
||||
const redirectUri = resolveBrowserOAuthRedirectUri("antigravity", "http://[::1]:20128/callback", {
|
||||
NEXT_PUBLIC_BASE_URL: "https://omniroute.example.com",
|
||||
ANTIGRAVITY_OAUTH_CLIENT_ID: "custom-antigravity.apps.googleusercontent.com",
|
||||
ANTIGRAVITY_OAUTH_CLIENT_SECRET: "custom-antigravity-secret",
|
||||
});
|
||||
const redirectUri = resolveBrowserOAuthRedirectUri(
|
||||
"antigravity",
|
||||
"http://[::1]:20128/callback",
|
||||
publicBaseEnv
|
||||
);
|
||||
|
||||
assert.equal(redirectUri, "https://omniroute.example.com/callback");
|
||||
});
|
||||
|
||||
test("custom Google OAuth callbacks default root loopback paths to callback path", () => {
|
||||
const redirectUri = resolveBrowserOAuthRedirectUri("antigravity", "http://127.0.0.1:20128", {
|
||||
NEXT_PUBLIC_BASE_URL: "https://omniroute.example.com",
|
||||
ANTIGRAVITY_OAUTH_CLIENT_ID: "custom-antigravity.apps.googleusercontent.com",
|
||||
ANTIGRAVITY_OAUTH_CLIENT_SECRET: "custom-antigravity-secret",
|
||||
});
|
||||
const redirectUri = resolveBrowserOAuthRedirectUri(
|
||||
"antigravity",
|
||||
"http://127.0.0.1:20128",
|
||||
publicBaseEnv
|
||||
);
|
||||
|
||||
assert.equal(redirectUri, "https://omniroute.example.com/callback");
|
||||
});
|
||||
|
||||
@@ -24,6 +24,18 @@ import {
|
||||
isNamedOpenAIStyleProvider,
|
||||
} from "../../src/app/api/providers/[id]/models/discovery/providerSets.ts";
|
||||
import { PROVIDER_MODELS_CONFIG } from "../../src/app/api/providers/[id]/models/discovery/providerModelsConfig.ts";
|
||||
import {
|
||||
buildCodexModelsUrl,
|
||||
CODEX_GITHUB_MODELS_URL,
|
||||
CODEX_MODELS_URL,
|
||||
clearCodexGithubCatalogCacheForTests,
|
||||
enrichCodexModelsFromGithubCatalog,
|
||||
fetchCodexDiscoveryModels,
|
||||
fetchCodexGithubCatalogModels,
|
||||
mergeCodexLiveModelsWithLocalCatalog,
|
||||
normalizeCodexGithubCatalogResponse,
|
||||
normalizeCodexModelsResponse,
|
||||
} from "../../src/app/api/providers/[id]/models/discovery/codex.ts";
|
||||
|
||||
// ── helpers leaf ─────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -132,6 +144,347 @@ test("providerModelsConfig aimlapi.parseResponse keeps only chat-completion mode
|
||||
assert.deepEqual(parsed, [{ id: "chat-1", name: "Chat 1" }]);
|
||||
});
|
||||
|
||||
// ── codex discovery leaf ────────────────────────────────────────────────────
|
||||
|
||||
test.beforeEach(() => {
|
||||
clearCodexGithubCatalogCacheForTests();
|
||||
});
|
||||
|
||||
test("codex.normalizeCodexModelsResponse maps data/models/map payloads to response models", () => {
|
||||
assert.deepEqual(normalizeCodexModelsResponse({ data: [{ id: "gpt-5.5", name: "GPT 5.5" }] }), [
|
||||
{
|
||||
id: "gpt-5.5",
|
||||
name: "GPT 5.5",
|
||||
owned_by: "codex",
|
||||
apiFormat: "responses",
|
||||
supportedEndpoints: ["responses"],
|
||||
},
|
||||
]);
|
||||
|
||||
assert.deepEqual(
|
||||
normalizeCodexModelsResponse({
|
||||
"gpt-5.4": { title: "GPT 5.4" },
|
||||
invalid: null,
|
||||
}).map((m) => ({ id: m.id, name: m.name })),
|
||||
[{ id: "gpt-5.4", name: "GPT 5.4" }]
|
||||
);
|
||||
});
|
||||
|
||||
test("codex.normalizeCodexModelsResponse parses the Codex live catalog shape", () => {
|
||||
const parsed = normalizeCodexModelsResponse({
|
||||
models: [
|
||||
{
|
||||
slug: "codex-auto-review",
|
||||
display_name: "Codex Auto Review",
|
||||
visibility: "hide",
|
||||
supported_in_api: true,
|
||||
},
|
||||
{
|
||||
slug: "gpt-5.4",
|
||||
display_name: "GPT-5.4",
|
||||
visibility: "list",
|
||||
supported_in_api: true,
|
||||
context_length: 400000,
|
||||
max_output_tokens: 128000,
|
||||
service_tiers: [{ id: "priority", name: "Fast" }],
|
||||
additional_speed_tiers: ["fast"],
|
||||
},
|
||||
{
|
||||
slug: "gpt-5.5",
|
||||
display_name: "GPT-5.5",
|
||||
visibility: "list",
|
||||
supported_in_api: true,
|
||||
max_input_tokens: 272000,
|
||||
top_provider: { max_completion_tokens: 64000 },
|
||||
},
|
||||
{
|
||||
slug: "internal-only",
|
||||
display_name: "Internal Only",
|
||||
visibility: "list",
|
||||
supported_in_api: false,
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
assert.deepEqual(
|
||||
parsed.map((m) => ({ id: m.id, name: m.name })),
|
||||
[
|
||||
{ id: "gpt-5.4", name: "GPT-5.4" },
|
||||
{ id: "gpt-5.5", name: "GPT-5.5" },
|
||||
]
|
||||
);
|
||||
assert.equal(parsed.find((model) => model.id === "gpt-5.4")?.inputTokenLimit, 400000);
|
||||
assert.equal(parsed.find((model) => model.id === "gpt-5.4")?.outputTokenLimit, 128000);
|
||||
assert.equal(parsed.find((model) => model.id === "gpt-5.5")?.inputTokenLimit, 272000);
|
||||
assert.equal(parsed.find((model) => model.id === "gpt-5.5")?.outputTokenLimit, 64000);
|
||||
});
|
||||
|
||||
test("codex.normalizeCodexGithubCatalogResponse parses current client catalog metadata", () => {
|
||||
const parsed = normalizeCodexGithubCatalogResponse({
|
||||
models: [
|
||||
{
|
||||
slug: "gpt-5.6-sol",
|
||||
display_name: "GPT-5.6-Sol",
|
||||
description: "Latest frontier agentic coding model.",
|
||||
visibility: "list",
|
||||
supported_in_api: true,
|
||||
minimal_client_version: "0.142.0",
|
||||
context_window: 372000,
|
||||
input_modalities: ["text", "image"],
|
||||
supported_reasoning_levels: [{ effort: "low" }, { effort: "ultra" }],
|
||||
},
|
||||
{
|
||||
slug: "future-model",
|
||||
display_name: "Future Model",
|
||||
visibility: "list",
|
||||
supported_in_api: true,
|
||||
minimal_client_version: "999.0.0",
|
||||
},
|
||||
{
|
||||
slug: "codex-auto-review",
|
||||
display_name: "Codex Auto Review",
|
||||
visibility: "hide",
|
||||
supported_in_api: true,
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
assert.deepEqual(
|
||||
parsed.map((model) => model.id),
|
||||
["gpt-5.6-sol"]
|
||||
);
|
||||
assert.equal(parsed[0]?.description, "Latest frontier agentic coding model.");
|
||||
assert.equal(parsed[0]?.inputTokenLimit, 372000);
|
||||
assert.equal(parsed[0]?.supportsThinking, true);
|
||||
assert.equal(parsed[0]?.supportsVision, true);
|
||||
});
|
||||
|
||||
test("codex.enrichCodexModelsFromGithubCatalog keeps live entitlement list authoritative", () => {
|
||||
const enriched = enrichCodexModelsFromGithubCatalog(
|
||||
[
|
||||
{
|
||||
id: "gpt-5.6-sol",
|
||||
name: "Live Sol",
|
||||
owned_by: "codex",
|
||||
apiFormat: "responses",
|
||||
supportedEndpoints: ["responses"],
|
||||
},
|
||||
],
|
||||
[
|
||||
{
|
||||
id: "gpt-5.6-sol",
|
||||
name: "GitHub Sol",
|
||||
owned_by: "codex",
|
||||
apiFormat: "responses",
|
||||
supportedEndpoints: ["responses"],
|
||||
inputTokenLimit: 372000,
|
||||
supportsVision: true,
|
||||
},
|
||||
{
|
||||
id: "gpt-5.6-luna",
|
||||
name: "GitHub Luna",
|
||||
owned_by: "codex",
|
||||
apiFormat: "responses",
|
||||
supportedEndpoints: ["responses"],
|
||||
},
|
||||
]
|
||||
);
|
||||
|
||||
assert.deepEqual(
|
||||
enriched.map((model) => model.id),
|
||||
["gpt-5.6-sol"]
|
||||
);
|
||||
assert.equal(enriched[0]?.name, "Live Sol");
|
||||
assert.equal(enriched[0]?.inputTokenLimit, 372000);
|
||||
assert.equal(enriched[0]?.supportsVision, true);
|
||||
});
|
||||
|
||||
test("codex.mergeCodexLiveModelsWithLocalCatalog preserves static effort aliases", () => {
|
||||
const merged = mergeCodexLiveModelsWithLocalCatalog(
|
||||
[
|
||||
{
|
||||
id: "gpt-5.6",
|
||||
name: "GPT 5.6 Live",
|
||||
owned_by: "codex",
|
||||
apiFormat: "responses",
|
||||
supportedEndpoints: ["responses"],
|
||||
inputTokenLimit: 512000,
|
||||
},
|
||||
],
|
||||
[
|
||||
{ id: "gpt-5.6", name: "GPT 5.6 Static", contextLength: 400000 },
|
||||
{ id: "gpt-5.6-low", name: "GPT 5.6 Low", contextLength: 400000 },
|
||||
]
|
||||
);
|
||||
|
||||
assert.deepEqual(
|
||||
merged.map((model) => model.id),
|
||||
["gpt-5.6", "gpt-5.6-low"]
|
||||
);
|
||||
assert.equal(merged.find((model) => model.id === "gpt-5.6")?.name, "GPT 5.6 Live");
|
||||
assert.equal(merged.find((model) => model.id === "gpt-5.6")?.inputTokenLimit, 512000);
|
||||
assert.equal(merged.find((model) => model.id === "gpt-5.6-low")?.inputTokenLimit, 400000);
|
||||
});
|
||||
|
||||
test("codex.normalizeCodexModelsResponse drops entries without an id", () => {
|
||||
const parsed = normalizeCodexModelsResponse({ models: [{ name: "" }, { model: "gpt-5.4" }] });
|
||||
assert.deepEqual(
|
||||
parsed.map((m) => m.id),
|
||||
["gpt-5.4"]
|
||||
);
|
||||
});
|
||||
|
||||
test("codex.fetchCodexDiscoveryModels returns null for missing token, auth failure, empty, or network error", async () => {
|
||||
assert.equal(
|
||||
await fetchCodexDiscoveryModels({
|
||||
accessToken: null,
|
||||
fetchImpl: async () => Response.json({ data: [{ id: "never-called" }] }),
|
||||
}),
|
||||
null
|
||||
);
|
||||
|
||||
assert.equal(
|
||||
await fetchCodexDiscoveryModels({
|
||||
accessToken: "tok",
|
||||
fetchImpl: async () => new Response("unauthorized", { status: 401 }),
|
||||
}),
|
||||
null
|
||||
);
|
||||
|
||||
assert.equal(
|
||||
await fetchCodexDiscoveryModels({
|
||||
accessToken: "tok",
|
||||
fetchImpl: async () => Response.json({ data: [] }),
|
||||
}),
|
||||
null
|
||||
);
|
||||
|
||||
assert.equal(
|
||||
await fetchCodexDiscoveryModels({
|
||||
accessToken: "tok",
|
||||
fetchImpl: async () => {
|
||||
throw new Error("network down");
|
||||
},
|
||||
}),
|
||||
null
|
||||
);
|
||||
});
|
||||
|
||||
test("codex.fetchCodexDiscoveryModels calls the Codex models endpoint with Codex bearer headers", async () => {
|
||||
let seenUrl = "";
|
||||
let seenAuthorization = "";
|
||||
let seenWorkspace = "";
|
||||
let seenOriginator = "";
|
||||
const models = await fetchCodexDiscoveryModels({
|
||||
accessToken: "codex-access",
|
||||
providerSpecificData: { chatgptAccountId: "account-123" },
|
||||
fetchImpl: async (url, init) => {
|
||||
seenUrl = url;
|
||||
seenAuthorization = init.headers.Authorization;
|
||||
seenWorkspace = init.headers["chatgpt-account-id"];
|
||||
seenOriginator = init.headers.originator;
|
||||
return Response.json({ models: [{ slug: "gpt-5.6", display_name: "GPT 5.6" }] });
|
||||
},
|
||||
});
|
||||
|
||||
assert.equal(CODEX_MODELS_URL, "https://chatgpt.com/backend-api/codex/models");
|
||||
assert.equal(seenUrl, buildCodexModelsUrl());
|
||||
assert.equal(seenAuthorization, "Bearer codex-access");
|
||||
assert.equal(seenWorkspace, "account-123");
|
||||
assert.equal(seenOriginator, "codex_cli_rs");
|
||||
assert.deepEqual(
|
||||
models?.map((m) => m.id),
|
||||
["gpt-5.6"]
|
||||
);
|
||||
});
|
||||
|
||||
test("codex.fetchCodexGithubCatalogModels fetches the OpenAI Codex repo catalog", async () => {
|
||||
let seenUrl = "";
|
||||
const models = await fetchCodexGithubCatalogModels({
|
||||
fetchImpl: async (url) => {
|
||||
seenUrl = url;
|
||||
return Response.json({
|
||||
models: [
|
||||
{
|
||||
slug: "gpt-5.6-terra",
|
||||
display_name: "GPT-5.6-Terra",
|
||||
visibility: "list",
|
||||
supported_in_api: true,
|
||||
minimal_client_version: "0.142.0",
|
||||
},
|
||||
],
|
||||
});
|
||||
},
|
||||
});
|
||||
|
||||
assert.equal(
|
||||
CODEX_GITHUB_MODELS_URL,
|
||||
"https://raw.githubusercontent.com/openai/codex/refs/heads/main/codex-rs/models-manager/models.json"
|
||||
);
|
||||
assert.equal(seenUrl, CODEX_GITHUB_MODELS_URL);
|
||||
assert.deepEqual(
|
||||
models?.map((model) => model.id),
|
||||
["gpt-5.6-terra"]
|
||||
);
|
||||
});
|
||||
|
||||
test("codex.fetchCodexGithubCatalogModels reuses cached catalog with ETags", async () => {
|
||||
const calls: Array<{ url: string; ifNoneMatch?: string }> = [];
|
||||
const first = await fetchCodexGithubCatalogModels({
|
||||
now: 1000,
|
||||
cacheTtlMs: 100,
|
||||
fetchImpl: async (url, init) => {
|
||||
calls.push({ url, ifNoneMatch: init.headers["If-None-Match"] });
|
||||
return Response.json(
|
||||
{
|
||||
models: [
|
||||
{
|
||||
slug: "gpt-5.6-luna",
|
||||
display_name: "GPT-5.6-Luna",
|
||||
visibility: "list",
|
||||
supported_in_api: true,
|
||||
minimal_client_version: "0.142.0",
|
||||
},
|
||||
],
|
||||
},
|
||||
{ headers: { etag: "catalog-v1" } }
|
||||
);
|
||||
},
|
||||
});
|
||||
const second = await fetchCodexGithubCatalogModels({
|
||||
now: 1050,
|
||||
cacheTtlMs: 100,
|
||||
fetchImpl: async () => {
|
||||
throw new Error("cache hit should not fetch");
|
||||
},
|
||||
});
|
||||
const third = await fetchCodexGithubCatalogModels({
|
||||
now: 1200,
|
||||
cacheTtlMs: 100,
|
||||
fetchImpl: async (url, init) => {
|
||||
calls.push({ url, ifNoneMatch: init.headers["If-None-Match"] });
|
||||
return new Response(null, { status: 304 });
|
||||
},
|
||||
});
|
||||
|
||||
assert.deepEqual(
|
||||
first?.map((model) => model.id),
|
||||
["gpt-5.6-luna"]
|
||||
);
|
||||
assert.deepEqual(
|
||||
second?.map((model) => model.id),
|
||||
["gpt-5.6-luna"]
|
||||
);
|
||||
assert.deepEqual(
|
||||
third?.map((model) => model.id),
|
||||
["gpt-5.6-luna"]
|
||||
);
|
||||
assert.deepEqual(calls, [
|
||||
{ url: CODEX_GITHUB_MODELS_URL, ifNoneMatch: undefined },
|
||||
{ url: CODEX_GITHUB_MODELS_URL, ifNoneMatch: "catalog-v1" },
|
||||
]);
|
||||
});
|
||||
|
||||
// ── host wiring guard ────────────────────────────────────────────────────────
|
||||
|
||||
test("route.ts imports the discovery leaves and no longer declares the moved consts", () => {
|
||||
@@ -139,7 +492,7 @@ test("route.ts imports the discovery leaves and no longer declares the moved con
|
||||
path.join("src", "app", "api", "providers", "[id]", "models", "route.ts"),
|
||||
"utf-8"
|
||||
);
|
||||
for (const leaf of ["helpers", "normalizers", "providerSets", "providerModelsConfig"]) {
|
||||
for (const leaf of ["helpers", "normalizers", "providerSets", "providerModelsConfig", "codex"]) {
|
||||
assert.match(
|
||||
route,
|
||||
new RegExp(`from "\\./discovery/${leaf}"`),
|
||||
|
||||
284
tests/unit/provider-models-route-codex.test.ts
Normal file
284
tests/unit/provider-models-route-codex.test.ts
Normal file
@@ -0,0 +1,284 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
const TEST_DATA_DIR = fs.mkdtempSync(
|
||||
path.join(os.tmpdir(), "omniroute-provider-model-routes-codex-")
|
||||
);
|
||||
process.env.DATA_DIR = TEST_DATA_DIR;
|
||||
|
||||
const core = await import("../../src/lib/db/core.ts");
|
||||
const providersDb = await import("../../src/lib/db/providers.ts");
|
||||
const modelsDb = await import("../../src/lib/db/models.ts");
|
||||
const providerModelsRoute = await import("../../src/app/api/providers/[id]/models/route.ts");
|
||||
const codexDiscovery = await import("../../src/app/api/providers/[id]/models/discovery/codex.ts");
|
||||
|
||||
type RouteModel = {
|
||||
id: string;
|
||||
name?: string;
|
||||
apiFormat?: string;
|
||||
supportedEndpoints?: string[];
|
||||
inputTokenLimit?: number;
|
||||
outputTokenLimit?: number;
|
||||
[key: string]: unknown;
|
||||
};
|
||||
|
||||
type RouteBody = {
|
||||
provider?: string;
|
||||
models?: RouteModel[];
|
||||
source?: string;
|
||||
warning?: string;
|
||||
intentional?: boolean;
|
||||
};
|
||||
|
||||
type ProviderOverrides = {
|
||||
authType?: string;
|
||||
apiKey?: string | null;
|
||||
accessToken?: string | null;
|
||||
providerSpecificData?: Record<string, unknown>;
|
||||
};
|
||||
|
||||
const originalFetch = globalThis.fetch;
|
||||
|
||||
async function resetStorage() {
|
||||
globalThis.fetch = originalFetch;
|
||||
codexDiscovery.clearCodexGithubCatalogCacheForTests();
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
fs.mkdirSync(TEST_DATA_DIR, { recursive: true });
|
||||
}
|
||||
|
||||
async function seedCodexConnection(overrides: ProviderOverrides = {}) {
|
||||
return providersDb.createProviderConnection({
|
||||
provider: "codex",
|
||||
authType: overrides.authType || "oauth",
|
||||
name: `codex-${Math.random().toString(16).slice(2, 8)}`,
|
||||
apiKey: overrides.apiKey,
|
||||
accessToken: overrides.accessToken,
|
||||
isActive: true,
|
||||
testStatus: "active",
|
||||
providerSpecificData: overrides.providerSpecificData || {},
|
||||
});
|
||||
}
|
||||
|
||||
async function callRoute(connectionId: string, search = "") {
|
||||
return providerModelsRoute.GET(
|
||||
new Request(`http://localhost/api/providers/${connectionId}/models${search}`),
|
||||
{ params: { id: connectionId } }
|
||||
);
|
||||
}
|
||||
|
||||
test.beforeEach(async () => {
|
||||
await resetStorage();
|
||||
});
|
||||
|
||||
test.after(async () => {
|
||||
globalThis.fetch = originalFetch;
|
||||
codexDiscovery.clearCodexGithubCatalogCacheForTests();
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test("provider models route discovers live Codex models and preserves static aliases", async () => {
|
||||
const connection = await seedCodexConnection({
|
||||
accessToken: "codex-access-token",
|
||||
providerSpecificData: { chatgptAccountId: "account-123" },
|
||||
});
|
||||
await modelsDb.replaceSyncedAvailableModelsForConnection("codex", connection.id, [
|
||||
{ id: "stale-codex-model", name: "Stale Codex Model", source: "imported" },
|
||||
]);
|
||||
const seenRequests: Array<Record<string, string | null>> = [];
|
||||
|
||||
globalThis.fetch = async (url, init) => {
|
||||
const requestUrl = String(url);
|
||||
const headers = new Headers(init?.headers as HeadersInit | undefined);
|
||||
seenRequests.push({
|
||||
url: requestUrl,
|
||||
authorization: headers.get("authorization"),
|
||||
workspaceId: headers.get("chatgpt-account-id"),
|
||||
originator: headers.get("originator"),
|
||||
userAgent: headers.get("user-agent"),
|
||||
});
|
||||
if (requestUrl.includes("raw.githubusercontent.com/openai/codex")) {
|
||||
return Response.json({
|
||||
models: [
|
||||
{
|
||||
slug: "gpt-5.6",
|
||||
display_name: "GPT 5.6 GitHub",
|
||||
visibility: "list",
|
||||
supported_in_api: true,
|
||||
minimal_client_version: "0.142.0",
|
||||
context_window: 372000,
|
||||
input_modalities: ["text", "image"],
|
||||
supported_reasoning_levels: [{ effort: "low" }, { effort: "high" }],
|
||||
},
|
||||
],
|
||||
});
|
||||
}
|
||||
return Response.json({
|
||||
models: [
|
||||
{ slug: "codex-auto-review", visibility: "hide", supported_in_api: true },
|
||||
{
|
||||
slug: "gpt-5.6",
|
||||
display_name: "GPT 5.6",
|
||||
visibility: "list",
|
||||
supported_in_api: true,
|
||||
max_input_tokens: 272000,
|
||||
max_output_tokens: 128000,
|
||||
},
|
||||
{ id: "", name: "missing-id" },
|
||||
],
|
||||
});
|
||||
};
|
||||
|
||||
const response = await callRoute(connection.id, "?refresh=true");
|
||||
const body = (await response.json()) as RouteBody;
|
||||
const modelIds = new Set(body.models?.map((model) => model.id));
|
||||
const liveModel = body.models?.find((model) => model.id === "gpt-5.6");
|
||||
const syncedModels = await modelsDb.getSyncedAvailableModelsForConnection("codex", connection.id);
|
||||
const syncedIds = new Set(syncedModels.map((model) => model.id));
|
||||
|
||||
assert.equal(response.status, 200);
|
||||
assert.equal(body.provider, "codex");
|
||||
assert.equal(body.source, "api");
|
||||
assert.deepEqual(seenRequests, [
|
||||
{
|
||||
url: "https://chatgpt.com/backend-api/codex/models?client_version=0.142.0",
|
||||
authorization: "Bearer codex-access-token",
|
||||
workspaceId: "account-123",
|
||||
originator: "codex_cli_rs",
|
||||
userAgent: "codex-cli/0.142.0 (Windows 10.0.26200; x64)",
|
||||
},
|
||||
{
|
||||
url: "https://raw.githubusercontent.com/openai/codex/refs/heads/main/codex-rs/models-manager/models.json",
|
||||
authorization: null,
|
||||
workspaceId: null,
|
||||
originator: null,
|
||||
userAgent: null,
|
||||
},
|
||||
]);
|
||||
assert.equal(liveModel?.name, "GPT 5.6");
|
||||
assert.equal(liveModel?.inputTokenLimit, 272000);
|
||||
assert.equal(liveModel?.outputTokenLimit, 128000);
|
||||
assert.equal(liveModel?.apiFormat, "responses");
|
||||
assert.deepEqual(liveModel?.supportedEndpoints, ["responses"]);
|
||||
assert.equal(liveModel?.supportsThinking, true);
|
||||
assert.equal(liveModel?.supportsVision, true);
|
||||
assert.ok(modelIds.has("gpt-5.5-low"));
|
||||
assert.ok(modelIds.has("gpt-5.4-xhigh"));
|
||||
assert.ok(syncedIds.has("gpt-5.6"));
|
||||
assert.ok(syncedIds.has("gpt-5.5-low"));
|
||||
assert.ok(syncedIds.has("gpt-5.4-xhigh"));
|
||||
assert.equal(modelIds.has("stale-codex-model"), false);
|
||||
assert.equal(syncedIds.has("stale-codex-model"), false);
|
||||
});
|
||||
|
||||
test("provider models route uses the GitHub Codex catalog when live discovery fails", async () => {
|
||||
const connection = await seedCodexConnection({ accessToken: "codex-access-token" });
|
||||
const seenUrls: string[] = [];
|
||||
|
||||
globalThis.fetch = async (url) => {
|
||||
const requestUrl = String(url);
|
||||
seenUrls.push(requestUrl);
|
||||
if (requestUrl.includes("raw.githubusercontent.com/openai/codex")) {
|
||||
return Response.json({
|
||||
models: [
|
||||
{
|
||||
slug: "gpt-5.6-sol",
|
||||
display_name: "GPT-5.6-Sol",
|
||||
visibility: "list",
|
||||
supported_in_api: true,
|
||||
minimal_client_version: "0.142.0",
|
||||
context_window: 372000,
|
||||
},
|
||||
],
|
||||
});
|
||||
}
|
||||
return new Response("upstream unavailable", { status: 503 });
|
||||
};
|
||||
|
||||
const response = await callRoute(connection.id, "?refresh=true");
|
||||
const body = (await response.json()) as RouteBody;
|
||||
const modelIds = new Set((body.models || []).map((model) => model.id));
|
||||
|
||||
assert.equal(response.status, 200);
|
||||
assert.equal(body.provider, "codex");
|
||||
assert.equal(body.source, "api");
|
||||
assert.equal(body.intentional, undefined);
|
||||
assert.equal(body.warning, "Codex live catalog unavailable — using GitHub model catalog");
|
||||
assert.ok(seenUrls.some((url) => url.includes("backend-api/codex/models")));
|
||||
assert.ok(seenUrls.some((url) => url.includes("raw.githubusercontent.com/openai/codex")));
|
||||
assert.ok(modelIds.has("gpt-5.6-sol"));
|
||||
assert.ok(modelIds.has("gpt-5.5-low"));
|
||||
});
|
||||
|
||||
test("provider models route returns cached Codex models when refresh discovery fails", async () => {
|
||||
const connection = await seedCodexConnection({ accessToken: "codex-access-token" });
|
||||
await modelsDb.replaceSyncedAvailableModelsForConnection("codex", connection.id, [
|
||||
{
|
||||
id: "cached-live-codex",
|
||||
name: "Cached Live Codex",
|
||||
source: "imported",
|
||||
apiFormat: "responses",
|
||||
supportedEndpoints: ["responses"],
|
||||
},
|
||||
]);
|
||||
|
||||
globalThis.fetch = async () => new Response("upstream unavailable", { status: 503 });
|
||||
|
||||
const response = await callRoute(connection.id, "?refresh=true");
|
||||
const body = (await response.json()) as RouteBody;
|
||||
|
||||
assert.equal(response.status, 200);
|
||||
assert.equal(body.provider, "codex");
|
||||
assert.equal(body.source, "cache");
|
||||
assert.equal(body.warning, "Codex live catalog unavailable — using cached catalog");
|
||||
assert.deepEqual(body.models, [
|
||||
{
|
||||
id: "cached-live-codex",
|
||||
name: "Cached Live Codex",
|
||||
source: "imported",
|
||||
apiFormat: "responses",
|
||||
supportedEndpoints: ["responses"],
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
test("provider models route falls back to local Codex catalog when live and GitHub fail", async () => {
|
||||
const connection = await seedCodexConnection({ accessToken: "codex-access-token" });
|
||||
|
||||
globalThis.fetch = async () => new Response("upstream unavailable", { status: 503 });
|
||||
|
||||
const response = await callRoute(connection.id, "?refresh=true");
|
||||
const body = (await response.json()) as RouteBody;
|
||||
|
||||
assert.equal(response.status, 200);
|
||||
assert.equal(body.provider, "codex");
|
||||
assert.equal(body.source, "local_catalog");
|
||||
assert.equal(body.intentional, true);
|
||||
assert.equal(body.warning, "Codex live and GitHub catalogs unavailable — using local catalog");
|
||||
assert.ok(body.models?.some((model) => model.id === "gpt-5.5"));
|
||||
});
|
||||
|
||||
test("provider models route returns codex gpt-5.4 effort variants when auto-fetch is disabled", async () => {
|
||||
const connection = await seedCodexConnection({
|
||||
apiKey: null,
|
||||
accessToken: "codex-access",
|
||||
providerSpecificData: { autoFetchModels: false },
|
||||
});
|
||||
|
||||
const response = await callRoute(connection.id);
|
||||
const body = (await response.json()) as RouteBody;
|
||||
const modelIds = new Set((body.models || []).map((model) => model.id));
|
||||
|
||||
assert.equal(response.status, 200);
|
||||
assert.equal(body.provider, "codex");
|
||||
assert.equal(body.source, "local_catalog");
|
||||
assert.ok(modelIds.has("gpt-5.4"));
|
||||
assert.ok(modelIds.has("gpt-5.4-low"));
|
||||
assert.ok(modelIds.has("gpt-5.4-medium"));
|
||||
assert.ok(modelIds.has("gpt-5.4-high"));
|
||||
assert.ok(modelIds.has("gpt-5.4-xhigh"));
|
||||
});
|
||||
@@ -690,27 +690,6 @@ test("provider models route returns the updated local catalog for GitHub Copilot
|
||||
);
|
||||
});
|
||||
|
||||
test("provider models route returns codex gpt-5.4 effort variants in the local catalog", async () => {
|
||||
const connection = await seedConnection("codex", {
|
||||
authType: "oauth",
|
||||
apiKey: null,
|
||||
accessToken: "codex-access",
|
||||
});
|
||||
|
||||
const response = await callRoute(connection.id);
|
||||
const body = (await response.json()) as any;
|
||||
const modelIds = new Set((body.models || []).map((model: any) => model.id));
|
||||
|
||||
assert.equal(response.status, 200);
|
||||
assert.equal(body.provider, "codex");
|
||||
assert.equal(body.source, "local_catalog");
|
||||
assert.ok(modelIds.has("gpt-5.4"));
|
||||
assert.ok(modelIds.has("gpt-5.4-low"));
|
||||
assert.ok(modelIds.has("gpt-5.4-medium"));
|
||||
assert.ok(modelIds.has("gpt-5.4-high"));
|
||||
assert.ok(modelIds.has("gpt-5.4-xhigh"));
|
||||
});
|
||||
|
||||
test("provider models route returns the expanded local catalog for Kiro", async () => {
|
||||
const connection = await seedConnection("kiro", {
|
||||
authType: "oauth",
|
||||
|
||||
92
tests/unit/sidebar-search-filter.test.ts
Normal file
92
tests/unit/sidebar-search-filter.test.ts
Normal file
@@ -0,0 +1,92 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { filterSidebarSectionsByQuery } from "../../src/shared/utils/sidebarSearch.ts";
|
||||
|
||||
type Item = { id: string; label: string };
|
||||
type Group = { type: "group"; id: string; items: Item[] };
|
||||
type Section = { id: string; children: (Item | Group)[] };
|
||||
|
||||
function makeSections(): Section[] {
|
||||
return [
|
||||
{
|
||||
id: "omni-proxy",
|
||||
children: [
|
||||
{ id: "combos", label: "Combos" },
|
||||
{ id: "providers", label: "Providers" },
|
||||
{
|
||||
type: "group",
|
||||
id: "tools",
|
||||
items: [
|
||||
{ id: "playground", label: "Playground" },
|
||||
{ id: "logs", label: "Request Logs" },
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
id: "configuration",
|
||||
children: [
|
||||
{ id: "settings", label: "Settings" },
|
||||
{ id: "webhooks", label: "Webhooks" },
|
||||
],
|
||||
},
|
||||
];
|
||||
}
|
||||
|
||||
test("empty/whitespace query returns all sections unchanged (identity-ish)", () => {
|
||||
const sections = makeSections();
|
||||
assert.deepEqual(filterSidebarSectionsByQuery(sections, ""), sections);
|
||||
assert.deepEqual(filterSidebarSectionsByQuery(sections, " "), sections);
|
||||
});
|
||||
|
||||
test("filters flat items by case-insensitive substring match on label", () => {
|
||||
const result = filterSidebarSectionsByQuery(makeSections(), "combo");
|
||||
assert.equal(result.length, 1);
|
||||
assert.equal(result[0].id, "omni-proxy");
|
||||
assert.deepEqual(
|
||||
result[0].children.map((c) => ("label" in c ? c.id : c.id)),
|
||||
["combos"]
|
||||
);
|
||||
});
|
||||
|
||||
test("matches are case-insensitive", () => {
|
||||
const result = filterSidebarSectionsByQuery(makeSections(), "SETTINGS");
|
||||
assert.equal(result.length, 1);
|
||||
assert.equal(result[0].id, "configuration");
|
||||
});
|
||||
|
||||
test("filters items inside a group, dropping non-matching group members", () => {
|
||||
const result = filterSidebarSectionsByQuery(makeSections(), "logs");
|
||||
assert.equal(result.length, 1);
|
||||
const group = result[0].children.find((c) => "type" in c && c.type === "group") as
|
||||
| Group
|
||||
| undefined;
|
||||
assert.ok(group, "expected the tools group to survive filtering");
|
||||
assert.deepEqual(
|
||||
group!.items.map((i) => i.id),
|
||||
["logs"]
|
||||
);
|
||||
});
|
||||
|
||||
test("drops a group entirely when none of its items match", () => {
|
||||
const result = filterSidebarSectionsByQuery(makeSections(), "playground-xyz-no-match");
|
||||
assert.deepEqual(result, []);
|
||||
});
|
||||
|
||||
test("drops sections that have no matching children at all", () => {
|
||||
const result = filterSidebarSectionsByQuery(makeSections(), "webhooks");
|
||||
assert.equal(result.length, 1);
|
||||
assert.equal(result[0].id, "configuration");
|
||||
});
|
||||
|
||||
test("query matching nothing returns an empty array", () => {
|
||||
const result = filterSidebarSectionsByQuery(makeSections(), "zzz-nonexistent-zzz");
|
||||
assert.deepEqual(result, []);
|
||||
});
|
||||
|
||||
test("does not mutate the input sections", () => {
|
||||
const sections = makeSections();
|
||||
const snapshot = JSON.parse(JSON.stringify(sections));
|
||||
filterSidebarSectionsByQuery(sections, "combo");
|
||||
assert.deepEqual(sections, snapshot);
|
||||
});
|
||||
@@ -19,6 +19,15 @@ test("isDegradedLocalCatalog: intentional local-only catalog is NOT a degraded f
|
||||
);
|
||||
// Case-insensitive on source.
|
||||
assert.equal(isDegradedLocalCatalog({ source: "LOCAL_CATALOG", intentional: true }), false);
|
||||
assert.equal(
|
||||
isDegradedLocalCatalog({
|
||||
source: "local_catalog",
|
||||
intentional: true,
|
||||
warning: "Codex live and GitHub catalogs unavailable — using local catalog",
|
||||
}),
|
||||
false,
|
||||
"Codex's intentional local fallback must import normally"
|
||||
);
|
||||
});
|
||||
|
||||
test("isDegradedLocalCatalog: a degraded remote-fetch fallback IS a failure (502)", () => {
|
||||
|
||||
@@ -255,3 +255,61 @@ test("OpenAI -> Claude (non-passthrough): built-in web_search IS still converted
|
||||
const toolNames = tools.map((t) => (t.function ? t.function.name : t.name));
|
||||
assert.ok(toolNames.includes(OMNIROUTE_WEB_SEARCH_FALLBACK_TOOL_NAME));
|
||||
});
|
||||
|
||||
// ── #3384: per-model interceptSearch override wins over every native-bypass default ──
|
||||
|
||||
test("#3384 interceptSearchOverride=true forces interception even on the Claude->Claude bypass path", () => {
|
||||
assert.equal(
|
||||
supportsNativeWebSearchFallbackBypass({
|
||||
provider: "claude",
|
||||
sourceFormat: "claude",
|
||||
targetFormat: "claude",
|
||||
nativeCodexPassthrough: false,
|
||||
interceptSearchOverride: true,
|
||||
}),
|
||||
false,
|
||||
"explicit interceptSearch:true must NOT bypass, overriding the native Claude passthrough"
|
||||
);
|
||||
});
|
||||
|
||||
test("#3384 interceptSearchOverride=false forces native passthrough even for a standard provider", () => {
|
||||
assert.equal(
|
||||
supportsNativeWebSearchFallbackBypass({
|
||||
provider: "openai",
|
||||
sourceFormat: "openai",
|
||||
targetFormat: "openai",
|
||||
nativeCodexPassthrough: false,
|
||||
interceptSearchOverride: false,
|
||||
}),
|
||||
true,
|
||||
"explicit interceptSearch:false must bypass even though OpenAI->OpenAI has no native default bypass"
|
||||
);
|
||||
});
|
||||
|
||||
test("#3384 interceptSearchOverride=undefined falls through to the existing native-bypass defaults", () => {
|
||||
assert.equal(
|
||||
supportsNativeWebSearchFallbackBypass({
|
||||
provider: "claude",
|
||||
sourceFormat: "claude",
|
||||
targetFormat: "claude",
|
||||
nativeCodexPassthrough: false,
|
||||
interceptSearchOverride: undefined,
|
||||
}),
|
||||
true,
|
||||
"no override configured — default Claude->Claude bypass still applies"
|
||||
);
|
||||
});
|
||||
|
||||
test("#3384 end-to-end: interceptSearchOverride=true converts the tool on the Claude->Claude bypass path", () => {
|
||||
const inputBody = { tools: [{ type: "web_search" }] };
|
||||
const { fallback } = prepareWebSearchFallbackBody(inputBody, {
|
||||
provider: "claude",
|
||||
sourceFormat: "claude",
|
||||
targetFormat: "claude",
|
||||
nativeCodexPassthrough: false,
|
||||
interceptSearchOverride: true,
|
||||
});
|
||||
|
||||
assert.equal(fallback.enabled, true);
|
||||
assert.equal(fallback.toolName, OMNIROUTE_WEB_SEARCH_FALLBACK_TOOL_NAME);
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user