diff --git a/AGENTS.md b/AGENTS.md index 14a0012e78..9279079fa0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -46,7 +46,7 @@ Repository map and Reference Documentation sections below. ## Project at a Glance -**OmniRoute** — unified AI proxy/router. One endpoint, 291 LLM providers, auto-fallback. +**OmniRoute** — unified AI proxy/router. One endpoint, 338 LLM providers, auto-fallback. | Layer | Location | Purpose | | ------------- | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | @@ -56,9 +56,9 @@ Repository map and Reference Documentation sections below. | Translators | `open-sse/translator/` | Format conversion (OpenAI↔Claude↔Gemini) | | Transformer | `open-sse/transformer/` | Responses API ↔ Chat Completions | | Services | `open-sse/services/` | Combo routing, rate limits, caching, etc | -| Database | `src/lib/db/` | SQLite domain modules (130 migrations) | +| Database | `src/lib/db/` | SQLite domain modules (144 migrations) | | Domain/Policy | `src/domain/` | Policy engine, cost rules, fallback logic | -| MCP Server | `open-sse/mcp-server/` | 105 tools (42 base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules), 3 transports (stdio / SSE / Streamable HTTP), 31 scopes | +| MCP Server | `open-sse/mcp-server/` | 105 tools (43 base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules), 3 transports (stdio / SSE / Streamable HTTP), 31 scopes | | A2A Server | `src/lib/a2a/` | JSON-RPC 2.0 agent protocol | | Skills | `src/lib/skills/` | Extensible skill framework | | Memory | `src/lib/memory/` | Persistent conversational memory | @@ -83,7 +83,7 @@ Client → /v1/chat/completions (Next.js route) API routes follow a consistent pattern: `Route → CORS preflight → Zod body validation → Optional auth (extractApiKey/isValidApiKey) → API key policy enforcement → Handler delegation (open-sse)`. No global Next.js middleware — interception is route-specific. -**Combo routing** (`open-sse/services/combo.ts`): 19 public strategies (priority, weighted, fill-first, round-robin, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline). Each target calls `handleSingleModel()` which wraps `handleChatCore()` with per-target error handling and circuit breaker checks. The `fusion` strategy is the exception: it fans out to a panel of models in parallel, then a judge model synthesizes one final answer (`open-sse/services/fusion.ts`). See `docs/routing/AUTO-COMBO.md` for the 13-factor Auto-Combo scoring + the full strategy table and `docs/architecture/RESILIENCE_GUIDE.md` for the 3 resilience layers. +**Combo routing** (`open-sse/services/combo.ts`): 19 public strategies (priority, weighted, fill-first, round-robin, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline). Each target calls `handleSingleModel()` which wraps `handleChatCore()` with per-target error handling and circuit breaker checks. The `fusion` strategy is the exception: it fans out to a panel of models in parallel, then a judge model synthesizes one final answer (`open-sse/services/fusion.ts`). See `docs/routing/AUTO-COMBO.md` for the 14-factor Auto-Combo scoring + the full strategy table and `docs/architecture/RESILIENCE_GUIDE.md` for the 3 resilience layers. --- @@ -404,7 +404,7 @@ For any non-trivial change, read the matching deep-dive first: | Repo navigation | `docs/architecture/REPOSITORY_MAP.md` | | Architecture | `docs/architecture/ARCHITECTURE.md` | | Engineering reference | `docs/architecture/CODEBASE_DOCUMENTATION.md` | -| Auto-Combo (13-factor scoring, 19 strategies) | `docs/routing/AUTO-COMBO.md` | +| Auto-Combo (14-factor scoring, 19 strategies) | `docs/routing/AUTO-COMBO.md` | | Resilience (3 mechanisms) | `docs/architecture/RESILIENCE_GUIDE.md` | | Reasoning replay | `docs/routing/REASONING_REPLAY.md` | | Skills framework | `docs/frameworks/SKILLS.md` | @@ -428,7 +428,7 @@ For any non-trivial change, read the matching deep-dive first: | Electron desktop app | `docs/guides/ELECTRON_GUIDE.md` | | Release flow | `docs/ops/RELEASE_CHECKLIST.md` | | Embedded services | `docs/frameworks/EMBEDDED-SERVICES.md` | -| Quality gates (~48 scripts, allowlist policy) | `docs/architecture/QUALITY_GATES.md` | +| Quality gates (~80 scripts, allowlist policy) | `docs/architecture/QUALITY_GATES.md` | --- @@ -620,7 +620,7 @@ focused checks, and use a Conventional Commit message (for example, `docs: slim ## Quality Gates & Ratchets -OmniRoute has **~48 quality-gate scripts** (`scripts/check/` + `scripts/quality/`) wired +OmniRoute has **~80 quality-gate scripts** (`scripts/check/` + `scripts/quality/`) wired across **9 gate-running jobs** in `.github/workflows/ci.yml` (`lint`, `quality-gate`, `quality-extended`, `docs-sync-strict`, `i18n-ui-coverage`, `i18n`, `pr-test-policy`, `test-vitest`, `sonarqube`), plus the `quality.yml` fast-gates job (PR→`release/**`) and diff --git a/README.md b/README.md index bf436620e3..85ee5512c7 100644 --- a/README.md +++ b/README.md @@ -7,19 +7,19 @@ # 🚀 OmniRoute — The Free AI Gateway -OmniRoute — Never stop coding. Every AI tool → 291 providers — 90+ free — through one endpoint. Claude Code, Codex, Cursor, Cline, Copilot & Antigravity into FREE Claude / GPT / Gemini with auto-fallback. RTK + Caveman stacked compression saves 15–95% tokens (~89% avg) — never hit limits. 291 AI providers · 90+ free tiers · ~1.53B free tokens/mo · 19 routing strategies · $0 to start. +OmniRoute — Never stop coding. Every AI tool → 338 providers — 90+ free — through one endpoint. Claude Code, Codex, Cursor, Cline, Copilot & Antigravity into FREE Claude / GPT / Gemini with auto-fallback. RTK + Caveman stacked compression saves 15–95% tokens (~89% avg) — never hit limits. 338 AI providers · 90+ free tiers · ~1.51B free tokens/mo · 19 routing strategies · $0 to start.
-## 💰 ~1.53B Free Tokens / Month +## 💰 ~1.51B Free Tokens / Month
-> Stacking free tiers by hand is painful — dozens of SDKs, dozens of rate limits, and no idea how much you actually have. OmniRoute aggregates the **documented** free tiers of **43 provider pools / 516 models** into one honest number and shows it live on the dashboard (`/dashboard/free-tiers`). +> Stacking free tiers by hand is painful — dozens of SDKs, dozens of rate limits, and no idea how much you actually have. OmniRoute aggregates the **documented** free tiers of **42 provider pools / 495 models** into one honest number and shows it live on the dashboard (`/dashboard/free-tiers`). -OmniRoute free-tier budget card: ~1.53B free tokens per month steady, up to ~2.15B in the first month with signup credits, from the documented free tiers of 43 provider pools / 516 models behind one endpoint. Honest pool-deduped math — each shared pool counted once (counting every rate limit 24/7 would read ~10B; not published), 15 providers ToS-flagged so you decide. Budget bar of the countable free pools with per-model grid (Mistral Large 3 1B, GPT-4o mini 150M, Gemini 2.5 Flash 60M … Claude Sonnet 4.5 25K), one-time first-month signup credits (vertex 300M, agentrouter 200M, predibase 25M, together 25M, glm-cn 20M, doubao 15M, ai21 10M, longcat 10M, deepseek 5M, hyperbolic 5M, nscale 5M), plus permanently-free no-token-cap providers (SiliconFlow, Z.AI GLM-Flash, Kilo, OpenCode Zen, baidu …) and a $10 OpenRouter top-up unlocking +24M/mo — surfaced separately so they never inflate the headline. Live used/remaining on /dashboard/free-tiers. +OmniRoute free-tier budget card: ~1.51B free tokens per month steady, up to ~2.13B in the first month with signup credits, from the documented free tiers of 42 provider pools / 495 models behind one endpoint. Honest pool-deduped math — each shared pool counted once (counting every rate limit 24/7 would read ~10B; not published), 15 providers ToS-flagged so you decide. Budget bar of the countable free pools with per-model grid (Mistral Large 3 1B, GPT-4o mini 150M, Gemini 2.5 Flash 60M … Claude Sonnet 4.5 25K), one-time first-month signup credits (vertex 300M, agentrouter 200M, predibase 25M, together 25M, glm-cn 20M, doubao 15M, ai21 10M, longcat 10M, deepseek 5M, hyperbolic 5M, nscale 5M), plus permanently-free no-token-cap providers (SiliconFlow, Z.AI GLM-Flash, Kilo, OpenCode Zen, baidu …) and a $10 OpenRouter top-up unlocking +24M/mo — surfaced separately so they never inflate the headline. Live used/remaining on /dashboard/free-tiers. > Animated summary of the live `/dashboard/free-tiers` page. Full methodology (pool dedupe, credit tiers, provider terms): **[docs/reference/FREE_TIERS.md](docs/reference/FREE_TIERS.md)**. > @@ -81,7 +81,7 @@ ⚙️ Features 🎯 Combos - 🌐 Providers + 🌐 Providers 🔌 CLI & MCP @@ -190,7 +190,7 @@ curl http://localhost:20128/v1/chat/completions \ -The Promise — One endpoint. 291 providers. Never stop building — OmniRoute picks the cheapest one that works. Six pillars: Never hit limits (auto-fallback across 291 providers in milliseconds, zero downtime) · Save up to 95% tokens (RTK + Caveman stacked compression cuts 15–95%, ~89% avg on tool-heavy sessions) · $0 to start (90+ free tiers, 40+ free forever — no card needed) · Every tool works (33 coding agents through one config) · One endpoint (OpenAI ↔ Claude ↔ Gemini ↔ Responses API at /v1) · Production-grade (circuit breakers, TLS stealth, MCP 105 tools, A2A, memory, guardrails, evals — 25,000+ tests). +The Promise — One endpoint. 338 providers. Never stop building — OmniRoute picks the cheapest one that works. Six pillars: Never hit limits (auto-fallback across 338 providers in milliseconds, zero downtime) · Save up to 95% tokens (RTK + Caveman stacked compression cuts 15–95%, ~89% avg on tool-heavy sessions) · $0 to start (90+ free tiers, 40+ free forever — no card needed) · Every tool works (33 coding agents through one config) · One endpoint (OpenAI ↔ Claude ↔ Gemini ↔ Responses API at /v1) · Production-grade (circuit breakers, TLS stealth, MCP 105 tools, A2A, memory, guardrails, evals — 25,000+ tests).

@@ -409,7 +409,7 @@ All **19** strategies — mix & match per combo step: 17 auto - 12-factor live scoring across every connection 🤖 + 14-factor live scoring across every connection 🤖 18 @@ -423,7 +423,7 @@ All **19** strategies — mix & match per combo step: -The Auto-Combo engine scores every candidate on **12 factors** (health, quota, cost, latency, success rate, freshness…) — see [`docs/routing/AUTO-COMBO.md`](docs/routing/AUTO-COMBO.md). +The Auto-Combo engine scores every candidate on **14 factors** (health, quota, cost, latency, success rate, freshness…) — see [`docs/routing/AUTO-COMBO.md`](docs/routing/AUTO-COMBO.md). ## @@ -441,7 +441,7 @@ All **19** strategies — mix & match per combo step: -What sets OmniRoute apart — comparison table vs 9router, OpenRouter, CLIProxyAPI and LiteLLM across 13 capabilities. OmniRoute: 291 providers, 90+ free providers built-in, 19 routing strategies, 12-engine token compression, built-in MCP server with 105 tools, A2A agent protocol, persistent memory, guardrails, cloud agents, TLS fingerprint stealth, Desktop/Termux/PWA, 43 i18n UI locales, 100% MIT self-hosted. OmniRoute is the only one with the full set; competitors show a mix of checks, partials and crosses. Verified from each project's docs. +What sets OmniRoute apart — comparison table vs 9router, OpenRouter, CLIProxyAPI and LiteLLM across 13 capabilities. OmniRoute: 338 providers, 90+ free providers built-in, 19 routing strategies, 12-engine token compression, built-in MCP server with 105 tools, A2A agent protocol, persistent memory, guardrails, cloud agents, TLS fingerprint stealth, Desktop/Termux/PWA, 43 i18n UI locales, 100% MIT self-hosted. OmniRoute is the only one with the full set; competitors show a mix of checks, partials and crosses. Verified from each project's docs. 📊 Full methodology & per-feature detail vs 9router, OpenRouter, CLIProxyAPI & LiteLLM → [`docs/comparison/OMNIROUTE_VS_ALTERNATIVES.md`](docs/comparison/OMNIROUTE_VS_ALTERNATIVES.md) @@ -499,8 +499,11 @@ Pix copia-e-cola: -> Recent highlights from **v3.8.20 → v3.8.49**. Full history in [`CHANGELOG.md`](CHANGELOG.md). +> Recent highlights from **v3.8.20 → v3.8.50**. Full history in [`CHANGELOG.md`](CHANGELOG.md). +- **🎛️ OmniConductor** — inbound A2A delegation to your agent fleet, Conductor skills on the Agent Card, and a dashboard panel with Faro push-to-talk voice chat. → [A2A Server](docs/frameworks/A2A-SERVER.md) +- **🛂 Adaptive admission & overload protection** — heavyweight chat requests queue instead of 503ing, with atomic RPM rolling leases per connection. → [Resilience Guide](docs/architecture/RESILIENCE_GUIDE.md) +- **🗂️ Canonical `/v1/models` ordering** — one contiguous provider-grouped block per provider (combos pinned first), stable across every catalog source. → [API Reference](docs/reference/API_REFERENCE.md) - **🗜️ Compression hardening** — default-on inflation guard, Caveman packs for DE / FR / JA + Chinese (wényán), RTK filters for Gradle & .NET. → [Compression](docs/compression/COMPRESSION_ENGINES.md) - **💸 Honest flat-rate cost** — subscription / coding-plan providers read **$0** in cost analytics; budget, quota & routing keep estimating. → [API Reference](docs/reference/API_REFERENCE.md) - **⚖️ Quota-Share routing** — split a shared account's quota fairly across pooled keys, work-conserving so idle slices are lent out. → [Resilience Guide](docs/architecture/RESILIENCE_GUIDE.md) @@ -515,7 +518,7 @@ Pix copia-e-cola: - **🖼️ New endpoints** — `/v1/ocr` (Mistral OCR) and `/v1/audio/translations` (Whisper-style) round out the media surface. → [API Reference](docs/reference/API_REFERENCE.md) - **🎨 Image / video / audio generation** — one API for media: xAI Grok Imagine & Novita AI video, ComfyUI, Freepik, Adobe Firefly, Microsoft Designer, Google Imagen, Segmind, EdgeTTS. → [API Reference](docs/reference/API_REFERENCE.md) - **🌍 Deployment & ops** — reverse-proxy `basePath`, browser-language auto-detect, per-key device tracking, root-less MITM trust, zh-TW localization. → [Environment](docs/reference/ENVIRONMENT.md) -- **🤝 More providers & agents** — Cursor Cloud Agent, Grok Build (xAI) with browser + OAuth login, Ollama first-class card, Claude Opus 5 & Sonnet 5, Kimi official partnership (Code/Web/Moonshot), Zed, Requesty, SenseNova, Yuanbao, Agnes AI… and a refreshed **291-provider catalog**. → [Providers](docs/reference/PROVIDER_REFERENCE.md) +- **🤝 More providers & agents** — Cursor Cloud Agent, Grok Build (xAI) with browser + OAuth login, Ollama first-class card, Claude Opus 5 & Sonnet 5, Kimi official partnership (Code/Web/Moonshot), Zed, Requesty, SenseNova, Yuanbao, Agnes AI… and a refreshed **338-provider catalog**. → [Providers](docs/reference/PROVIDER_REFERENCE.md) - **📡 Routing transparency** — every response carries an `X-OmniRoute-Decision` header naming the strategy/provider/latency that served it, a new `cache-optimized` combo strategy + Auto-Combo `cacheAffinity` factor route repeat requests back to the connection holding the cached prefix, and a read-only `/v1/auto-combo/{channel}/candidates` endpoint exposes an `auto/*` channel's live candidate pool. → [Auto-Combo](docs/routing/AUTO-COMBO.md) - **⚡ Local performance & infra** — one-click local Redis, Cloudflare Workers / Deno Deploy relay deployers, Bifrost & Mux as supervised embedded services. → [Embedded Services](docs/frameworks/EMBEDDED-SERVICES.md) @@ -576,11 +579,11 @@ Pix copia-e-cola:
-## 🌐 291 AI Providers — 90+ Free +## 🌐 338 AI Providers — 90+ Free
-> The most complete catalog of any open-source router: **291 providers**, **90+ with a free tier**, **40+ free forever**. +> The most complete catalog of any open-source router: **338 providers**, **90+ with a free tier**, **40+ free forever**.
@@ -1038,7 +1041,7 @@ same process on one port, so there is no separate CLI-only package today. RuntimeNode.js 22.x / 24.x LTS — >=22.22.2 <23 || >=24.0.0 <27 LanguageTypeScript 6.0 — 100% TypeScript across src/ and open-sse/ (zero any in core since v2.0) FrameworkNext.js 16 + React 19 + Tailwind CSS 4 - Databasebetter-sqlite3 (SQLite, WAL journaling) + LowDB (JSON legacy) — 95 domain modules, 110 migrations + Databasebetter-sqlite3 (SQLite, WAL journaling) + LowDB (JSON legacy) — 95 domain modules, 144 migrations MemorySQLite FTS5 full-text + int8-quantized vector embeddings, typed decay SchemasZod 4 — MCP tool I/O validation + API contracts ProtocolsMCP (stdio / HTTP / SSE) + A2A v0.3 (JSON-RPC 2.0 + SSE) @@ -1099,9 +1102,9 @@ same process on one port, so there is no separate CLI-only package today. Compression Rules FormatJSON rule-pack schemas for Caveman and RTK filters Compression Language PacksLanguage detection and Caveman rule-pack authoring Resilience GuideCircuit breakers, cooldowns, queue, anti-thundering herd, TLS spoofing - Auto-Combo Engine12-factor scoring, mode packs, self-healing + Auto-Combo Engine14-factor scoring, mode packs, self-healing Proxy Guide3-level proxy system, 1proxy marketplace, registry CRUD - Free Tiers25+ free API providers consolidated directory + Free Tiers90+ free providers consolidated directory (42 documented token pools / 495 models) Features GalleryVisual dashboard tour with screenshots Codebase DocumentationBeginner-friendly codebase walkthrough @@ -1112,7 +1115,7 @@ same process on one port, so there is no separate CLI-only package today. DocumentDescription API ReferenceAll endpoints with examples OpenAPI SpecOpenAPI 3.0 specification - MCP Server104 MCP tools, IDE configs, Python/TS/Go clients + MCP Server105 MCP tools, IDE configs, Python/TS/Go clients MCP Server GuideMCP installation, transports, and tool reference A2A ServerJSON-RPC 2.0 protocol, skills, streaming, task mgmt A2A Server GuideA2A agent card, tasks, skills, and streaming @@ -1126,7 +1129,7 @@ same process on one port, so there is no separate CLI-only package today. Branching & Release ModelWhere PRs target (release/*), what main and tags mean ChangelogFull per-version release history Security PolicyVulnerability reporting and security practices - i18n Guide40+ language support, translation workflow, RTL + i18n Guide43-language support, translation workflow, RTL Release ChecklistPre-release validation steps Coverage PlanTest coverage strategy and 25,000+ test suite @@ -1269,7 +1272,7 @@ A heartfelt thank-you to the people who fund OmniRoute out of their own pocket
-## 👥 500+ Contributors +## 👥 320+ Contributors
diff --git a/docs/README.md b/docs/README.md index 9bcf44845c..92935da7a1 100644 --- a/docs/README.md +++ b/docs/README.md @@ -121,7 +121,7 @@ Pluggable subsystems exposed to clients, agents, and operators. Combo routing, scoring, and replay. -- [AUTO-COMBO.md](routing/AUTO-COMBO.md) — Auto-Combo (multi-factor scoring, 17 strategies). +- [AUTO-COMBO.md](routing/AUTO-COMBO.md) — Auto-Combo (multi-factor scoring, 19 strategies). - [QUOTA_SHARE.md](routing/QUOTA_SHARE.md) — quota sharing engine. - [REASONING_REPLAY.md](routing/REASONING_REPLAY.md) — reasoning replay cache. - [REASONING_ROUTING.md](routing/REASONING_ROUTING.md) — reasoning routing rules (effort/budget rule engine). diff --git a/docs/architecture/ARCHITECTURE.md b/docs/architecture/ARCHITECTURE.md index e99e9b8262..b41bc81297 100644 --- a/docs/architecture/ARCHITECTURE.md +++ b/docs/architecture/ARCHITECTURE.md @@ -17,7 +17,7 @@ It provides a single OpenAI-compatible endpoint (`/v1/*`) and routes traffic acr Core capabilities: -- OpenAI-compatible API surface for CLI/tools (271 providers, 89 executors) +- OpenAI-compatible API surface for CLI/tools (338 providers, 101 executors) - Request/response translation across provider formats - Model combo fallback (multi-model sequence) - Structured combo steps (`provider + model + connection`) with runtime ordering by `compositeTiers` @@ -365,7 +365,7 @@ relying on a static combo definition. It powers the `auto/*` model prefix family Key capabilities: -- **17 routing strategies** (priority, weighted, fill-first, round-robin, P2C, random, +- **19 routing strategies** (priority, weighted, fill-first, round-robin, P2C, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, **auto**, lkgp, context-optimized, context-relay, **fusion**, plus a fallback path) — auto is the headline addition in v3.8.0; `fusion` (panel fan-out + judge synthesis, diff --git a/docs/architecture/CODEBASE_DOCUMENTATION.md b/docs/architecture/CODEBASE_DOCUMENTATION.md index 07544bef2c..e12317863b 100644 --- a/docs/architecture/CODEBASE_DOCUMENTATION.md +++ b/docs/architecture/CODEBASE_DOCUMENTATION.md @@ -452,12 +452,12 @@ open-sse/ ├── types.d.ts ├── config/ Provider registries, header profiles, identity, … ├── handlers/ Request handlers (chat, embeddings, audio, image, …) -├── executors/ 89 provider-specific HTTP executors +├── executors/ 101 provider-specific HTTP executors ├── translator/ Format conversion (OpenAI ↔ Claude ↔ Gemini ↔ Cursor ↔ Kiro) ├── transformer/ Responses API ↔ Chat Completions stream transformer ├── services/ 80+ service modules (combos, fallback, quotas, identity, …) ├── utils/ Streaming helpers, TLS client, AWS SigV4, proxy fetch, … -└── mcp-server/ MCP server (3 transports, 32 scopes, 99 tools) +└── mcp-server/ MCP server (3 transports, 31 scopes, 105 tools) ``` ### 4.1 `open-sse/handlers/` @@ -482,7 +482,7 @@ open-sse/ ### 4.2 `open-sse/executors/` -84 provider executors, each extending `BaseExecutor` (`base.ts`): +101 provider executors, each extending `BaseExecutor` (`base.ts`): `antigravity`, `azure-openai`, `blackbox-web`, `chatgpt-web`, `cliproxyapi`, `cloudflare-ai`, `codex`, `commandCode`, `cursor`, `default`, `devin-cli`, @@ -491,7 +491,7 @@ open-sse/ (shared identity helper) and `index.ts` (registry). > Note: providers not listed here are served by `default.ts` using the generic -> OpenAI-compatible executor. The full provider catalog (268 entries) lives in +> OpenAI-compatible executor. The full provider catalog (338 providers) lives in > `src/shared/constants/providers.ts`. ### 4.3 `open-sse/translator/` @@ -524,7 +524,7 @@ Highlights (full list under `open-sse/services/`): | Concern | Files | | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Combo routing | `combo.ts` (17 strategies), `comboConfig.ts`, `comboMetrics.ts`, `comboManifestMetrics.ts`, `comboAgentMiddleware.ts` | +| Combo routing | `combo.ts` (19 strategies), `comboConfig.ts`, `comboMetrics.ts`, `comboManifestMetrics.ts`, `comboAgentMiddleware.ts` | | Auto Combo engine | `autoCombo/` — `engine.ts`, `scoring.ts`, `taskFitness.ts`, `virtualFactory.ts`, `modePacks.ts`, `autoPrefix.ts`, `persistence.ts`, `providerDiversity.ts`, `providerRegistryAccessor.ts`, `routerStrategy.ts`, `selfHealing.ts`, `index.ts` | | Resilience | `accountFallback.ts` (cooldown + lockout), `errorClassifier.ts`, `emergencyFallback.ts`, `rateLimitManager.ts`, `rateLimitSemaphore.ts`, `accountSemaphore.ts`, `accountSelector.ts` | | Quotas | `quotaMonitor.ts`, `quotaPreflight.ts`, `bailianQuotaFetcher.ts`, `codexQuotaFetcher.ts`, `deepseekQuotaFetcher.ts`, `openrouterQuotaFetcher.ts`, `openrouterFreeWindow.ts`, `crofUsageFetcher.ts`, `antigravityCredits.ts` | @@ -544,7 +544,7 @@ Highlights (full list under `open-sse/services/`): 5 compression tools, 3 memory tools, 4 skills tools, plus advanced tools added through `advancedTools.ts`). - **3 transports**: stdio, HTTP Streamable, SSE. -- **13 scopes** declared in `src/shared/constants/mcpScopes.ts`. +- **31 scopes** declared in `src/shared/constants/mcpScopes.ts`. - Audit table: `mcp_tool_audit` (populated by `audit.ts`). - Files: `server.ts`, `index.ts`, `httpTransport.ts`, `audit.ts`, `scopeEnforcement.ts`, `runtimeHeartbeat.ts`, `descriptionCompressor.ts`, `schemas/{tools, a2a, audit, index}.ts`, diff --git a/docs/architecture/MONITORING_SECTIONS.md b/docs/architecture/MONITORING_SECTIONS.md index ca03a1136c..3fa9657caa 100644 --- a/docs/architecture/MONITORING_SECTIONS.md +++ b/docs/architecture/MONITORING_SECTIONS.md @@ -142,5 +142,5 @@ Namespaces added by Group B: | `sidebar.costsOverview` | Costs overview item | | `activity.*` | All Activity page strings (title, verbs, filters, empty state) | -Source-of-truth locales: `pt-BR` and `en`. All other 39 locales fall back to +Source-of-truth locales: `pt-BR` and `en`. All other 41 locales fall back to English via the `next-intl` fallback mechanism (configured in `src/i18n/config.ts`). diff --git a/docs/architecture/QUALITY_GATES.md b/docs/architecture/QUALITY_GATES.md index a7a3421cd7..63fbdea72f 100644 --- a/docs/architecture/QUALITY_GATES.md +++ b/docs/architecture/QUALITY_GATES.md @@ -207,7 +207,7 @@ Runs after `build`. Blocks merge on failure. | Suite | Validates | Blocking | | ---------------- | ------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | -| `test:vitest` | MCP server (94 tools), autoCombo, cache — vitest runner | Yes | +| `test:vitest` | MCP server (105 tools), autoCombo, cache — vitest runner | Yes | | `test:vitest:ui` | UI component tests — vitest runner | **Blocking** — pre-existing failures are explicitly excluded in `vitest.config.ts`; new failures fail the job | ### Nightly workflows (scheduled, advisory) diff --git a/docs/architecture/REPOSITORY_MAP.md b/docs/architecture/REPOSITORY_MAP.md index 20238334d1..5cee8f04a0 100644 --- a/docs/architecture/REPOSITORY_MAP.md +++ b/docs/architecture/REPOSITORY_MAP.md @@ -182,7 +182,7 @@ src/ | `compliance/` | Audit log + provider audit — see `docs/security/COMPLIANCE.md` | | `compression/` | Compression engine glue (engines live in `open-sse/services/compression/`) | | `config/` | Runtime config helpers | -| `db/` | 95+ domain DB modules + 110+ migrations (always go through here for SQLite) | +| `db/` | 95+ domain DB modules + 144 migrations (always go through here for SQLite) | | `quota/` | Quota Sharing Engine: `dimensions.ts` (types/Zod), `types.ts` (QuotaStore interface), `sqliteQuotaStore.ts`, `redisQuotaStore.ts`, `storeFactory.ts`, `fairShare.ts`, `burnRate.ts`, `planResolver.ts`, `planRegistry.ts`, `saturationSignals.ts`, `enforce.ts`, `spendRecorder.ts` — see `docs/routing/QUOTA_SHARE.md` | | `radar/` | Radar free-model catalog client: `feedSchema.ts`, `pinnedKeys.ts`, `verify.ts`, `sync.ts`, `applyFeed.ts`, `index.ts` (`getRadarCatalog()`) — see `docs/frameworks/RADAR.md` | | `display/` | UI formatting helpers (cost, latency, etc.) | @@ -209,7 +209,7 @@ src/ | `cacheLayer.ts`, `idempotencyLayer.ts` | Request caching + idempotency | | (~30 more top-level files) | Specialized helpers (logEnv, modelsDevSync, piiSanitizer, etc.) | -### `src/db/` — Database (94 modules + 106 migrations) +### `src/db/` — Database (117 modules + 144 migrations) | Subdir | Purpose | | ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | @@ -244,9 +244,9 @@ src/ | Module | Purpose | | -------------------------------- | ---------------------------------------------------------------------- | -| `constants/providers.ts` | **236 providers** with Zod validation (source of truth) | +| `constants/providers.ts` | **338 providers** with Zod validation (source of truth) | | `constants/cliTools.ts` | External CLI tool registry | -| `constants/routingStrategies.ts` | **17 routing strategies** with priorities | +| `constants/routingStrategies.ts` | **19 routing strategies** with priorities | | `constants/publicApiRoutes.ts` | Routes that require Bearer (vs management) auth | | `constants/upstreamHeaders.ts` | Header denylist for upstream requests | | `validation/schemas.ts` | ~80 Zod schemas (single source of truth for API contracts) | @@ -270,7 +270,7 @@ open-sse/ ├── translator/ # Format converters (9 request, 9 response, 9 helpers) ├── transformer/ # Responses API ↔ Chat Completions (TransformStream) ├── services/ # ~80+ service modules (combo, accountFallback, autoCombo, reasoningCache, claude code/chatgpt stealth, modelDeprecation, taskAwareRouter, workflowFSM, etc.) -├── mcp-server/ # MCP server (99 tools, 3 transports, 32 scopes) +├── mcp-server/ # MCP server (105 tools, 3 transports, 31 scopes) ├── config/ # Provider/model registries, header config, model aliases ├── utils/ # TLS client, proxy fetch/dispatcher, network helpers ├── index.ts # Workspace entry @@ -398,16 +398,16 @@ open-sse/ | `COVERAGE_PLAN.md` | Coverage goals and current state | | `FREE_TIERS.md` | Curated free-tier providers (48+ free + 11 OAuth) | | `CLI-TOOLS.md` | External CLI integrations + Internal OmniRoute CLI | -| `I18N.md` | i18n architecture, adding a language, 30 locales | +| `I18N.md` | i18n architecture, adding a language, 43 locales | | `UNINSTALL.md` | Clean uninstall steps | -| `PROVIDER_REFERENCE.md` | **Auto-generated** catalog of 236 providers (regen: `npm run gen:provider-reference`) | +| `PROVIDER_REFERENCE.md` | **Auto-generated** catalog of 338 providers (regen: `npm run gen:provider-reference`) | ### Subsystem deep-dives | Doc | Purpose | | -------------------------- | ------------------------------------------------------------------- | -| `MCP-SERVER.md` | MCP server: 99 tools, 3 transports, 32 scopes, REST endpoints | -| `A2A-SERVER.md` | A2A v0.3: JSON-RPC, 5 skills, REST helpers, agent card | +| `MCP-SERVER.md` | MCP server: 105 tools, 3 transports, 31 scopes, REST endpoints | +| `A2A-SERVER.md` | A2A v0.3: JSON-RPC, 6 skills, REST helpers, agent card | | `AGENT_PROTOCOLS_GUIDE.md` | Unified guide: A2A vs ACP vs Cloud Agents | | `CLOUD_AGENT.md` | Codex Cloud / Devin / Jules orchestration | | `SKILLS.md` | Skills framework (built-in + marketplace + SkillsSH + sandbox) | diff --git a/docs/diagrams/cli-terminal.svg b/docs/diagrams/cli-terminal.svg index bb015e3a12..989a0b8ff6 100644 --- a/docs/diagrams/cli-terminal.svg +++ b/docs/diagrams/cli-terminal.svg @@ -1,4 +1,4 @@ - + Compact animated terminal cycling three real OmniRoute CLI commands with a typewriter effect and a scrolling subcommand ticker; the first frame shows the completed providers-list screen. @@ -6,7 +6,7 @@ omniroute — 80+ commands -omniroute providers listOmniRoute Providers1f3a9c2e  anthropic   Claude Max 20x    active8c2d5b1a  codex       Codex Pro (team)  activef4e0a97b  glm         GLM Coding Plan   active03bd6e5f  kimi        Kimi K2 free      active… 264 more providers +omniroute providers listOmniRoute Providers1f3a9c2e  anthropic   Claude Max 20x    active8c2d5b1a  codex       Codex Pro (team)  activef4e0a97b  glm         GLM Coding Plan   active03bd6e5f  kimi        Kimi K2 free      active… 334 more providers $ omniroute providers list @@ -14,7 +14,7 @@ -OmniRoute Providers1f3a9c2e  anthropic   Claude Max 20x    active8c2d5b1a  codex       Codex Pro (team)  activef4e0a97b  glm         GLM Coding Plan   active03bd6e5f  kimi        Kimi K2 free      active… 264 more providers +OmniRoute Providers1f3a9c2e  anthropic   Claude Max 20x    active8c2d5b1a  codex       Codex Pro (team)  activef4e0a97b  glm         GLM Coding Plan   active03bd6e5f  kimi        Kimi K2 free      active… 334 more providers $ @@ -32,7 +32,7 @@ -OmniRoute Health  Status: healthy   Uptime: 4d 12h 33m  Requests (24h): 18,412   p95: 412ms  Breakers: ● 24 closed  ◒ 1 half-open  ○ 0 open  Providers: 278 registered   90+ free tiers… live: /dashboard · omniroute status +OmniRoute Health  Status: healthy   Uptime: 4d 12h 33m  Requests (24h): 18,412   p95: 412ms  Breakers: ● 24 closed  ◒ 1 half-open  ○ 0 open  Providers: 338 registered   90+ free tiers… live: /dashboard · omniroute status diff --git a/docs/diagrams/comparison-table.svg b/docs/diagrams/comparison-table.svg index 35bd075e64..509ec7f0ac 100644 --- a/docs/diagrams/comparison-table.svg +++ b/docs/diagrams/comparison-table.svg @@ -1,4 +1,4 @@ - + Static-header comparison table where each capability row fades in top to bottom; the OmniRoute column is highlighted and shows a check or a leading value in every row, while competitors show a mix of checks, partials and crosses. @@ -23,7 +23,7 @@ Providers - 290 + 338 40+ 400+* ~5 @@ -57,7 +57,7 @@ Built-in MCP server (own tools) - 104 + 105 diff --git a/docs/diagrams/free-tier-budget.svg b/docs/diagrams/free-tier-budget.svg index 6ae541be74..c3ae12e259 100644 --- a/docs/diagrams/free-tier-budget.svg +++ b/docs/diagrams/free-tier-budget.svg @@ -1,4 +1,4 @@ - + @@ -60,10 +60,10 @@ - ~1.53B + ~1.51B FREE TOKENS / MONTH · STEADY - up to ~2.15B in your first month — signup credits - documented free tiers · 43 provider pools · 460+ models · one endpoint + up to ~2.13B in your first month — signup credits + documented free tiers · 42 provider pools · 495 models · one endpoint @@ -74,7 +74,7 @@ every rate limit · 24/7 we don't publish that - ~1.53B + ~1.51B each shared free pool counted once ✓ 15 providers ToS-flagged — we flag it · you decide diff --git a/docs/diagrams/promise-pillars.svg b/docs/diagrams/promise-pillars.svg index 6f3a43632f..39d9ccffb4 100644 --- a/docs/diagrams/promise-pillars.svg +++ b/docs/diagrams/promise-pillars.svg @@ -1,4 +1,4 @@ - + Animated promise card: six pillar tiles fade in in reading order, then a soft colored border highlight sweeps from tile to tile in a continuous cycle. @@ -21,7 +21,7 @@ - One endpoint. 290 providers. Never stop building — OmniRoute picks the cheapest one that works. + One endpoint. 338 providers. Never stop building — OmniRoute picks the cheapest one that works. @@ -38,7 +38,7 @@ Never hit limits - Auto-fallback across 290 providers in + Auto-fallback across 338 providers in milliseconds. Quota out? The next provider takes over — zero downtime. @@ -125,7 +125,7 @@ Production-grade - Circuit breakers, TLS stealth, MCP (104 + Circuit breakers, TLS stealth, MCP (105 tools), A2A, memory, guardrails, evals — 25,000+ tests. diff --git a/docs/diagrams/readme-hero.svg b/docs/diagrams/readme-hero.svg index 58f8cb1d5b..ea615506ee 100644 --- a/docs/diagrams/readme-hero.svg +++ b/docs/diagrams/readme-hero.svg @@ -1,4 +1,4 @@ - + Animated hero card: a pulse travels the divider line and a compression bar demo repeatedly shrinks a prompt by up to 95 percent; all headline content is static and readable on the first frame. @@ -28,7 +28,7 @@ Never stop coding. - Every AI tool → 290 providers — 90+ free — through one endpoint. + Every AI tool → 338 providers — 90+ free — through one endpoint. Claude Code · Codex · Cursor · Cline · Copilot · Antigravity  →  FREE Claude / GPT / Gemini · auto-fallback @@ -66,13 +66,13 @@ - 290 + 338 AI PROVIDERS 90+ FREE TIERS - ~1.53B + ~1.51B FREE TOKENS / MO 15–95% diff --git a/docs/diagrams/tier-cascade.svg b/docs/diagrams/tier-cascade.svg index 4f819814fd..7944762286 100644 --- a/docs/diagrams/tier-cascade.svg +++ b/docs/diagrams/tier-cascade.svg @@ -89,7 +89,7 @@ OmniRoute — Smart Router - RTK + Caveman compression · 18 routing strategies + RTK + Caveman compression · 19 routing strategies Circuit breakers · TLS stealth · MCP · A2A · Guardrails diff --git a/docs/frameworks/ACP.md b/docs/frameworks/ACP.md index 3a7b386220..997fb828b0 100644 --- a/docs/frameworks/ACP.md +++ b/docs/frameworks/ACP.md @@ -543,7 +543,7 @@ const agents = detectInstalledAgents(); ## What's Next? - **[API Reference](../reference/API_REFERENCE.md)** — REST API endpoints -- **[Provider Reference](../reference/PROVIDER_REFERENCE.md)** — All 226 providers +- **[Provider Reference](../reference/PROVIDER_REFERENCE.md)** — All 338 providers - **[MCP Server](./MCP-SERVER.md)** — Model Context Protocol integration - **[A2A Server](./A2A-SERVER.md)** — Agent-to-Agent protocol - **[Cloud Agent](./CLOUD_AGENT.md)** — Cloud-based agents diff --git a/docs/frameworks/AGENTBRIDGE.md b/docs/frameworks/AGENTBRIDGE.md index 76949f46fe..85cba9a28c 100644 --- a/docs/frameworks/AGENTBRIDGE.md +++ b/docs/frameworks/AGENTBRIDGE.md @@ -22,7 +22,7 @@ When an IDE agent (e.g., GitHub Copilot, Cursor, Claude Code) makes an API call, This means you can: -- **Reroute any agent to any provider**: Copilot talking to OpenAI? Redirect it to Anthropic Claude, Gemini, or any of OmniRoute's 226+ providers. +- **Reroute any agent to any provider**: Copilot talking to OpenAI? Redirect it to Anthropic Claude, Gemini, or any of OmniRoute's 338 providers. - **Apply model mappings**: `gemini-3-flash` → `claude-sonnet-4.7` transparently at the handler level. - **Observe all agent traffic**: every intercepted request is published to the [Traffic Inspector](./TRAFFIC_INSPECTOR.md). - **Apply OmniRoute resilience**: combo routing, circuit breakers, fallbacks, and cost tracking work for IDE agent traffic too. diff --git a/docs/frameworks/AGENT_PROTOCOLS_GUIDE.md b/docs/frameworks/AGENT_PROTOCOLS_GUIDE.md index 305c398ced..46354225a5 100644 --- a/docs/frameworks/AGENT_PROTOCOLS_GUIDE.md +++ b/docs/frameworks/AGENT_PROTOCOLS_GUIDE.md @@ -183,7 +183,7 @@ Both have "long-running tasks" but at different layers: curl http://localhost:20128/.well-known/agent.json ``` -Returns the Agent Card with all 5 skills, transports, and version. +Returns the Agent Card with all 6 skills, transports, and version. ### Call OmniRoute as an A2A agent diff --git a/docs/frameworks/MCP-SERVER.md b/docs/frameworks/MCP-SERVER.md index 7933941011..b9ea9e6b36 100644 --- a/docs/frameworks/MCP-SERVER.md +++ b/docs/frameworks/MCP-SERVER.md @@ -6,7 +6,7 @@ lastUpdated: 2026-06-28 # OmniRoute MCP Server Documentation -> Model Context Protocol server with 104 tools across routing, cache, compression, memory, skills, proxy, pool, and context source operations. +> Model Context Protocol server with 105 tools across routing, cache, compression, memory, skills, proxy, pool, and context source operations. > > Source of truth: `open-sse/mcp-server/server.ts` computes **104 unique tools** with `countUniqueMcpTools()`: 42 canonical definitions (including the six CCR lifecycle tools and the agent-skills trio), plus memory (3), skills (4), GitHub skills (3), pool (6), gamification (8), plugins (8), Notion (6), Obsidian (22), and two RTK-only compression tools. @@ -369,7 +369,7 @@ MCP tool, prompt, and resource registries can compress descriptions at registrat Description compression shrinks each tool's metadata; **tool-cardinality reduction** goes one step further by reducing _how many_ tools are announced at all. Advertising fewer tools in the `tools/list` manifest cuts the per-request token cost the client's model pays for the tool catalog ("layer 5" compression). The implementation is a pure, stateless filter in `open-sse/mcp-server/toolCardinality.ts` (`reduceToolManifest`), wired into the registration loop in `createMcpServer()` (`open-sse/mcp-server/server.ts`). -**Opt-in, off by default.** The filter only runs when at least one of two environment variables is set; with neither set, all 104 tools are announced unchanged. +**Opt-in, off by default.** The filter only runs when at least one of two environment variables is set; with neither set, all 105 tools are announced unchanged. | Variable | Mode | | :--------------- | :-------------------------------------------------------------------------------------- | diff --git a/docs/frameworks/OPEN_SSE_ARCHITECTURE.md b/docs/frameworks/OPEN_SSE_ARCHITECTURE.md index dc525245dd..83ebf35a95 100644 --- a/docs/frameworks/OPEN_SSE_ARCHITECTURE.md +++ b/docs/frameworks/OPEN_SSE_ARCHITECTURE.md @@ -228,7 +228,7 @@ export async function handleComboChat(body, comboId): Promise { } ``` -Supports **17 routing strategies** (see `src/shared/constants/routingStrategies.ts`): +Supports **19 routing strategies** (see `src/shared/constants/routingStrategies.ts`): | Strategy | Behavior | | ------------------- | ------------------------------------------------------------------------- | @@ -252,7 +252,7 @@ Supports **17 routing strategies** (see `src/shared/constants/routingStrategies. ### base.ts (1170 LOC) -The **abstract executor** that all 67 executors extend. It contains: +The **abstract executor** that all 101 executors extend. It contains: - `buildUrl()` — default URL construction (subclasses override for custom) - `buildHeaders()` — default headers (auth, content-type) @@ -368,7 +368,7 @@ const result = await executor.execute({ }); ```` -The factory is generated from `config/providerRegistry.ts` which lists all 212+ providers and their executor class. +The factory is generated from `config/providerRegistry.ts` which lists all 338 providers and their executor class. --- @@ -408,7 +408,7 @@ Common translations: - **30+ tools** (provider management, combos, memory, cache, compression, 1proxy, skills) - **3 transports**: stdio, SSE, Streamable HTTP -- **13 scopes** for fine-grained authorization +- **31 scopes** for fine-grained authorization ### Tool Registration @@ -487,7 +487,7 @@ This handles: | File | Purpose | | ----------------------------- | --------------------------------- | -| `providerRegistry.ts` | 212+ provider definitions | +| `providerRegistry.ts` | 338 provider definitions | | `providerModels.ts` | Model aliases, format mapping | | `constants.ts` | Timeouts, limits, status codes | | `defaultThinkingSignature.ts` | Default Claude thinking signature | diff --git a/docs/guides/I18N.md b/docs/guides/I18N.md index 1cc5a7f088..87b06395de 100644 --- a/docs/guides/I18N.md +++ b/docs/guides/I18N.md @@ -88,9 +88,9 @@ README variants) are not yet handled by the new pipeline and are still used. ### Source of Truth - **UI strings**: `src/i18n/messages/en.json` (English source, ~2800 keys) -- **Locale files**: `src/i18n/messages/{locale}.json` (30 translations) +- **Locale files**: `src/i18n/messages/{locale}.json` (43 translations) - **Framework**: `next-intl` with cookie-based locale resolution -- **Config**: `src/i18n/config.ts` — defines all 30 locales, language names, flags +- **Config**: `src/i18n/config.ts` — defines all 43 locales, language names, flags ### Runtime Flow @@ -304,7 +304,7 @@ invocation. Use `config lang set` to persist. 43 locale files ship in `bin/cli/locales/`. Full translations: `en`, `pt-BR`. Scaffold-only (all keys fall back to `en`): `bn`, `gu`, `he`, `in`, `mr`, `ms`, `phi`, `sw`, `ta`, `te`, `ur`. -All other 29 locales have `common` + `program` keys translated. +All other 30 locales have `common` + `program` keys translated. ### Adding a new CLI locale @@ -533,7 +533,7 @@ python3 scripts/i18n/validate_translation.py quick -l '${{ matrix.lang }}' ``` src/i18n/ -├── config.ts # Locale definitions (30 locales, RTL config) +├── config.ts # Locale definitions (43 locales, RTL config) ├── request.ts # Runtime locale resolution └── messages/ ├── en.json # Source of truth (~2800 keys) diff --git a/docs/i18n/ar/llm.txt b/docs/i18n/ar/llm.txt index d157540caa..454f5dcd4d 100644 --- a/docs/i18n/ar/llm.txt +++ b/docs/i18n/ar/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/az/llm.txt b/docs/i18n/az/llm.txt index 075229caa5..d829905bcd 100644 --- a/docs/i18n/az/llm.txt +++ b/docs/i18n/az/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/bg/llm.txt b/docs/i18n/bg/llm.txt index 075229caa5..d829905bcd 100644 --- a/docs/i18n/bg/llm.txt +++ b/docs/i18n/bg/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/bn/llm.txt b/docs/i18n/bn/llm.txt index 686cf2afbc..e0a399b644 100644 --- a/docs/i18n/bn/llm.txt +++ b/docs/i18n/bn/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/cs/llm.txt b/docs/i18n/cs/llm.txt index 5dbaf03e99..922e176440 100644 --- a/docs/i18n/cs/llm.txt +++ b/docs/i18n/cs/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/da/llm.txt b/docs/i18n/da/llm.txt index 29093c0fce..fc4d0b2466 100644 --- a/docs/i18n/da/llm.txt +++ b/docs/i18n/da/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/de/llm.txt b/docs/i18n/de/llm.txt index 175fbf938b..c013784a40 100644 --- a/docs/i18n/de/llm.txt +++ b/docs/i18n/de/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/es/llm.txt b/docs/i18n/es/llm.txt index 01d19c8093..c383e19104 100644 --- a/docs/i18n/es/llm.txt +++ b/docs/i18n/es/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/fa/llm.txt b/docs/i18n/fa/llm.txt index 4ee1ea1f79..fe7f03f67d 100644 --- a/docs/i18n/fa/llm.txt +++ b/docs/i18n/fa/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/fi/llm.txt b/docs/i18n/fi/llm.txt index 777b728cb9..11273b2f23 100644 --- a/docs/i18n/fi/llm.txt +++ b/docs/i18n/fi/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/fr/llm.txt b/docs/i18n/fr/llm.txt index 83e126582a..e1c32578f8 100644 --- a/docs/i18n/fr/llm.txt +++ b/docs/i18n/fr/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/gu/llm.txt b/docs/i18n/gu/llm.txt index 909f1818f2..26a0640805 100644 --- a/docs/i18n/gu/llm.txt +++ b/docs/i18n/gu/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/he/llm.txt b/docs/i18n/he/llm.txt index a229eeb907..39c47704a3 100644 --- a/docs/i18n/he/llm.txt +++ b/docs/i18n/he/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/hi/llm.txt b/docs/i18n/hi/llm.txt index e92f5650b4..27fddd2e19 100644 --- a/docs/i18n/hi/llm.txt +++ b/docs/i18n/hi/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/hu/llm.txt b/docs/i18n/hu/llm.txt index 40b0dd095c..a1d63275bb 100644 --- a/docs/i18n/hu/llm.txt +++ b/docs/i18n/hu/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/id/llm.txt b/docs/i18n/id/llm.txt index 8b61036b68..377391d905 100644 --- a/docs/i18n/id/llm.txt +++ b/docs/i18n/id/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/in/llm.txt b/docs/i18n/in/llm.txt index 86bf32e43e..bbd35be114 100644 --- a/docs/i18n/in/llm.txt +++ b/docs/i18n/in/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/it/llm.txt b/docs/i18n/it/llm.txt index 0341b59136..d14785348e 100644 --- a/docs/i18n/it/llm.txt +++ b/docs/i18n/it/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/ja/llm.txt b/docs/i18n/ja/llm.txt index 92af0f6bae..e6c52a40c6 100644 --- a/docs/i18n/ja/llm.txt +++ b/docs/i18n/ja/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/ko/llm.txt b/docs/i18n/ko/llm.txt index 74258069db..e0e5de2910 100644 --- a/docs/i18n/ko/llm.txt +++ b/docs/i18n/ko/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/mr/llm.txt b/docs/i18n/mr/llm.txt index 9f79456c69..5a9ce36db4 100644 --- a/docs/i18n/mr/llm.txt +++ b/docs/i18n/mr/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/ms/llm.txt b/docs/i18n/ms/llm.txt index 9c5b6b9760..4c7cee1944 100644 --- a/docs/i18n/ms/llm.txt +++ b/docs/i18n/ms/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/nl/llm.txt b/docs/i18n/nl/llm.txt index 1b2b3435a3..77a8c8a29c 100644 --- a/docs/i18n/nl/llm.txt +++ b/docs/i18n/nl/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/no/llm.txt b/docs/i18n/no/llm.txt index f3f4e73e48..992be4b63f 100644 --- a/docs/i18n/no/llm.txt +++ b/docs/i18n/no/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/phi/llm.txt b/docs/i18n/phi/llm.txt index 015777774b..9a248f4be2 100644 --- a/docs/i18n/phi/llm.txt +++ b/docs/i18n/phi/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/pl/llm.txt b/docs/i18n/pl/llm.txt index b2dc9c9fbf..e47af25132 100644 --- a/docs/i18n/pl/llm.txt +++ b/docs/i18n/pl/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/pt-BR/llm.txt b/docs/i18n/pt-BR/llm.txt index c89a273b7d..2ea921f25e 100644 --- a/docs/i18n/pt-BR/llm.txt +++ b/docs/i18n/pt-BR/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/pt/llm.txt b/docs/i18n/pt/llm.txt index b29bbb630c..185f5badfe 100644 --- a/docs/i18n/pt/llm.txt +++ b/docs/i18n/pt/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/ro/llm.txt b/docs/i18n/ro/llm.txt index 1910de17e7..aa9f7446e3 100644 --- a/docs/i18n/ro/llm.txt +++ b/docs/i18n/ro/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/ru/llm.txt b/docs/i18n/ru/llm.txt index 4ed0d7286e..3f008ce8c7 100644 --- a/docs/i18n/ru/llm.txt +++ b/docs/i18n/ru/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/sk/llm.txt b/docs/i18n/sk/llm.txt index 8547faa1df..dda73b9e77 100644 --- a/docs/i18n/sk/llm.txt +++ b/docs/i18n/sk/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/sv/llm.txt b/docs/i18n/sv/llm.txt index 9c6c24b4db..9fcf0081fe 100644 --- a/docs/i18n/sv/llm.txt +++ b/docs/i18n/sv/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/sw/llm.txt b/docs/i18n/sw/llm.txt index 6549c8f5f3..69650fcc5c 100644 --- a/docs/i18n/sw/llm.txt +++ b/docs/i18n/sw/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/ta/llm.txt b/docs/i18n/ta/llm.txt index d1f7d8967f..f2660ef9a5 100644 --- a/docs/i18n/ta/llm.txt +++ b/docs/i18n/ta/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/te/llm.txt b/docs/i18n/te/llm.txt index bb2fee59c7..8e04cb2e9f 100644 --- a/docs/i18n/te/llm.txt +++ b/docs/i18n/te/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/th/llm.txt b/docs/i18n/th/llm.txt index 430442fdb2..2b6ae43aeb 100644 --- a/docs/i18n/th/llm.txt +++ b/docs/i18n/th/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/tr/llm.txt b/docs/i18n/tr/llm.txt index 91092cff9f..0c1c41eb05 100644 --- a/docs/i18n/tr/llm.txt +++ b/docs/i18n/tr/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/uk-UA/llm.txt b/docs/i18n/uk-UA/llm.txt index 7b70b0553a..5a7213ed4a 100644 --- a/docs/i18n/uk-UA/llm.txt +++ b/docs/i18n/uk-UA/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/ur/llm.txt b/docs/i18n/ur/llm.txt index fa189b93fa..95720495f8 100644 --- a/docs/i18n/ur/llm.txt +++ b/docs/i18n/ur/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/vi/llm.txt b/docs/i18n/vi/llm.txt index 21fdc5fc78..b4c9ccbe4d 100644 --- a/docs/i18n/vi/llm.txt +++ b/docs/i18n/vi/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/zh-CN/llm.txt b/docs/i18n/zh-CN/llm.txt index cdd5e17b39..22c88dc845 100644 --- a/docs/i18n/zh-CN/llm.txt +++ b/docs/i18n/zh-CN/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/zh-TW/llm.txt b/docs/i18n/zh-TW/llm.txt index e5a59d9617..13149ea8d1 100644 --- a/docs/i18n/zh-TW/llm.txt +++ b/docs/i18n/zh-TW/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/reference/PROVIDER_REFERENCE.md b/docs/reference/PROVIDER_REFERENCE.md index 56844c4999..613209bb49 100644 --- a/docs/reference/PROVIDER_REFERENCE.md +++ b/docs/reference/PROVIDER_REFERENCE.md @@ -1,20 +1,21 @@ --- title: "Provider Reference" version: 3.8.50 -lastUpdated: 2026-08-05 +lastUpdated: 2026-08-12 --- # Provider Reference > **Auto-generated** from `src/shared/constants/providers.ts` — do not edit by hand. > Regenerate with: `npm run gen:provider-reference` -> **Last generated:** 2026-08-05 +> **Last generated:** 2026-08-12 -Total providers: **291**. See category breakdown below. +Total providers: **338**. See category breakdown below. ## Categories - **Free** — free tier with API key (configured via dashboard) +- **No-auth** — public endpoints that require no key or sign-in at all - **OAuth** — sign-in flow handled by OmniRoute, no API key needed - **Web cookie** — wraps the provider's web app via cookie auth - **API key** — paid provider configured via API key (free credits may apply) @@ -33,7 +34,22 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each --- -## OAuth Providers (23) +## No-auth Providers (no key required) (10) + +| ID | Alias | Name | Tags | Website | Notes | Tool calling | +|----|-------|------|------|---------|-------|--------------| +| `aihorde` | `horde` | AI Horde | No-auth | [link](https://aihorde.net) | No API key required — uses AI Horde's documented anonymous key. Adding a free aihorde.net key is optional and only buys higher queue priority (kudos). | — | +| `auggie` | `aug` | Augment (Auggie CLI) | No-auth | [link](https://augmentcode.com) | No API key stored by OmniRoute. Install the Auggie CLI and run `auggie login` on this machine, then OmniRoute spawns it locally for each request. | — | +| `chipotle` | `pepper` | Chipotle Pepper AI (Free) | No-auth | [link](https://amelia.chipotle.com) | No credentials required. Uses Chipotle's public support chatbot via reverse-engineered SockJS/STOMP protocol. | — | +| `devin-cli-agentic` | `dva` | Devin CLI Agentic Bridge | No-auth | [link](https://docs.devin.ai/work-with-devin/devin-cli) | Authentication is owned by the official Devin CLI in its isolated bridge volume. | emulated | +| `duckduckgo-web` | `ddgw` | DuckDuckGo AI Chat | No-auth | [link](https://duckduckgo.com/duckchat) | No credentials required — DuckDuckGo AI Chat is anonymous and free. | emulated | +| `felo-web` | `felo` | Felo | No-auth | [link](https://felo.ai) | No credentials required — Felo is a free, no-signup chat/search aggregator. | — | +| `mimocode` | `mcode` | MiMoCode (Free) | No-auth | [link](https://mimo.mi.com) | No API key required. The executor auto-generates JWT tokens via device fingerprint bootstrap. | — | +| `opencode` | `oc` | OpenCode Free | No-auth | [link](https://opencode.ai) | No API key required — uses OpenCode's public free endpoint. | — | +| `theoldllm` | `tllm` | The Old LLM (Free) | No-auth | [link](https://theoldllm.vercel.app) | No credentials required. The executor auto-generates access tokens via an embedded Playwright browser instance. | — | +| `veoaifree-web` | `veo-free` | Veo AI Free | No-auth, video | [link](https://veoaifree.com) | No auth required. Rate limited to 6 requests/hour per IP. | — | + +## OAuth Providers (25) | ID | Alias | Name | Tags | Website | Notes | |----|-------|------|------|---------|-------| @@ -46,7 +62,7 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each | `codebuddy-cn` | `cbcn` | CodeBuddy CN | OAuth | [link](https://copilot.tencent.com) | Tencent CodeBuddy CN (copilot.tencent.com). Sign in via the official CLI device-code flow, or paste a direct API key (sent as Authorization: Bearer). Catalog: GLM / Kimi / MiniMax / DeepSeek / Hunyuan. | | `codex` | `cx` | OpenAI Codex | OAuth | — | — | | `cursor` | `cu` | Cursor IDE | OAuth | — | — | -| `devin-cli` | `dv` | Devin CLI (Official) | OAuth | [link](https://cli.devin.ai) | Requires the Devin CLI binary. Run `devin auth login` to authenticate, or provide your WINDSURF_API_KEY. Install: https://cli.devin.ai | +| `devin-cli` | `dv` | Devin CLI | OAuth | [link](https://cli.devin.ai) | Requires the Devin CLI binary. Run `devin auth login` to authenticate, or provide your WINDSURF_API_KEY. Install: https://cli.devin.ai | | `devin-desktop` | — | Devin Desktop | OAuth | [link](https://devin.ai) | Paste an existing Devin API key from an authenticated Devin session. Key export availability and steps vary by Devin version and account. | | `ghe-copilot` | `ghe-copilot` | GitHub Enterprise Copilot | OAuth | — | Enter your GHE instance URL (e.g., https://ghe.company.com) in provider settings, then authenticate via device flow. | | `github` | `gh` | GitHub Copilot | OAuth | — | — | @@ -55,13 +71,15 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each | `kilocode` | `kc` | Kilo Code | OAuth | — | — | | `kimi-coding` | `kmc` | Kimi Code CLI | OAuth | [link](https://www.kimi.com/code?aff=omniroute) | Sign in with the same Kimi account used by Kimi Code CLI. OmniRoute uses the CLI OAuth flow and Kimi Coding Plan endpoints. | | `kiro` | `kr` | Kiro AI | OAuth | — | Free tier: 50 credits/month (~25K–100K tokens). ⚠️ Kiro ToS prohibits third-party proxy/harness use. | +| `openference` | `of` | Openference | OAuth | [link](https://openference.com) | Sign in with your Openference account to route requests through api.openference.com. An active plan is required for inference — OAuth may authenticate but return 402 without one. | | `qoder` | `if` | Qoder | OAuth | — | — | +| `raycast` | `rc` | Raycast Pro AI | OAuth | [link](https://raycast.com/ai) | Unofficial integration — uses your Raycast Pro subscription via credentials from the macOS app (Auto-Import or manual capture). May break on Raycast updates. Not for redistribution; personal use only. | | `trae` | `tr` | Trae | OAuth | [link](https://trae.ai) | Trae is an AI-native IDE by ByteDance (SOLO remote agent). Authorize via trae.ai in the popup, or sign in at solo.trae.ai and paste the Cloud-IDE-JWT (sent as 'Authorization: Cloud-IDE-JWT ', ~14-day lifetime) as the access token; web_id/biz_user_id/user_unique_id/scope/tenant/region propagate via providerSpecificData. No headless refresh for pasted tokens — re-paste on expiry. | | `xai-oauth` | `xao` | xAI OAuth (Grok) | OAuth | [link](https://x.ai) | Sign in with xAI to use api.x.ai models such as Grok 4.5. This is separate from Grok Build JWT sessions, which use cli-chat-proxy.grok.com and grok-build model aliases. | | `zed` | `zd` | Zed IDE | OAuth | [link](https://zed.dev) | Zed stores LLM provider credentials (OpenAI, Anthropic, Google, Mistral, xAI) in the OS keychain. Use the Import button below to discover and import them automatically. | | `zed-hosted` | — | Zed Hosted Models | OAuth | [link](https://zed.dev) | Sign in with your Zed account (native-app sign-in). OmniRoute generates a one-time RSA keypair and opens zed.dev to authorize it — on a remote/headless install, copy the resulting 127.0.0.1 callback URL from your browser's address bar and paste it back here. Distinct from the 'Zed IDE' credential-import entry above: this proxies chat completions through Zed's own hosted model aggregator (cloud.zed.dev), fronting Anthropic/OpenAI/Google/xAI models under your Zed plan. | -## Web Cookie Providers (31) +## Web Cookie Providers (34) | ID | Alias | Name | Tags | Website | Notes | Tool calling | |----|-------|------|------|---------|-------|--------------| @@ -69,9 +87,11 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each | `adobe-firefly` | `firefly` | Adobe Firefly (Image/Video) | Web cookie | [link](https://firefly.adobe.com) | RECOMMENDED: firefly.adobe.com signed-in → F12 → Network → click firefly-3p.ff.adobe.io (generate-async or models/discovery) → Request Headers → Authorization → copy the token AFTER 'Bearer ' (starts with eyJ…). Cookie-only from firefly.adobe.com mints a GUEST token → 401/403; only multi-domain IMS cookies (adobelogin.com) or that Bearer JWT work. Unofficial/experimental media + Limits. | — | | `blackbox-web` | `bb-web` | Blackbox Web (Subscription) | Web cookie | [link](https://app.blackbox.ai) | Paste your __Secure-authjs.session-token value or full cookie header from app.blackbox.ai | emulated | | `chatgpt-web` | `cgpt-web` | ChatGPT Web (Plus/Pro) | Web cookie | [link](https://chatgpt.com) | Paste your __Secure-next-auth.session-token cookie value from chatgpt.com | emulated | +| `chatgpt-web-codex` | `cgpt-codex` | ChatGPT Web (Codex) | Web cookie | [link](https://chatgpt.com) | Paste the full ChatGPT Cookie header. OmniRoute verifies it in an isolated headless browser profile. | native | | `claude-web` | `cw` | Claude Web | Web cookie | [link](https://claude.ai) | Paste your session cookie from claude.ai | none | +| `conol-web` | `cnl` | Conol (Unofficial/Experimental) | Web cookie | [link](https://conol.ai) | Use browser sign-in, or paste the full Cookie header from conol.ai. The __Secure-better-auth.session_token cookie is required. | — | | `copilot-m365-web` | `m365copilot` | Microsoft 365 Copilot (BizChat) | Web cookie | [link](https://m365.cloud.microsoft/chat) | Sign in at m365.cloud.microsoft/chat, then open DevTools → Network → filter 'WS' → click the Chathub WebSocket connection. Copy both the access_token query parameter AND the account-specific Chathub path segment from its request URL (wss://…/Chathub/?…&access_token=…). It is NOT an Authorization: Bearer header on an XHR/Fetch request. The token is short-lived; this is an unofficial integration. | — | -| `copilot-web` | `copilot` | Microsoft Copilot Web | Web cookie | [link](https://copilot.microsoft.com) | Paste your access_token from copilot.microsoft.com (or export a .har file from DevTools while logged in) | — | +| `copilot-web` | `copilot` | Microsoft Copilot Web | Web cookie | [link](https://copilot.microsoft.com) | Paste the access_token from an authenticated copilot.microsoft.com request (DevTools → Network → Authorization), or export a HAR while logged in | — | | `deepseek-web` | `ds-web` | DeepSeek Web | Web cookie | [link](https://chat.deepseek.com) | Paste your userToken from chat.deepseek.com — DevTools → Application → Local Storage → userToken | emulated | | `doubao-web` | `db` | Dola Web (ByteDance) | Web cookie | [link](https://www.dola.com) | Paste the full Cookie header from www.dola.com. It should include sessionid, ttwid, and s_v_web_id. If s_v_web_id is unavailable, fp=verify_... from a chat/completion request URL can be used as a fallback. | — | | `gemini-business` | `gembiz` | Gemini Business (Enterprise) | Web cookie | [link](https://business.gemini.google) | From your enterprise account: open business.gemini.google/home/cid/{your-cid}, then copy __Secure-1PSID and __Secure-1PSIDTS cookies from DevTools → Application → Cookies. Paste as a cookie header below. | — | @@ -91,19 +111,20 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each | `promptql` | `pql` | PromptQL (Unofficial/Experimental) | Web cookie | [link](https://prompt.ql.app) | Paste the Bearer JWT from prompt.ql.app DevTools → Network → graphql → Authorization (token only). Optional projectId + session Cookie for refresh. | — | | `qwen-web` | `qwen-web` | Qwen Web (Free) | Web cookie | [link](https://chat.qwen.ai) | Open chat.qwen.ai, log in, then open DevTools → Application → Local Storage → copy the "token" value (or use tongyi_sso_ticket cookie as Bearer token). | emulated | | `t3-web` | `t3chat` | t3.chat (Pro/Free) | Web cookie | [link](https://t3.chat) | Open t3.chat in your browser, log in, then open DevTools → Application → Local Storage → https://t3.chat. Copy the value of 'convex-session-id'. Also open DevTools → Network, copy the Cookie header from any request. Paste both values here. See provider setup docs for a step-by-step guide. | emulated | +| `tinycms-web` | `tcw` | TinyCMS Web (Free/Sub) | Web cookie | [link](https://site.tinycms.xyz) | Go to site.tinycms.xyz, open DevTools → Application → Local Storage, copy the value of 'app-config-uuid' (starts with 'R'), and paste it here. | — | | `v0-vercel-web` | `v0-vercel-web` | v0 Vercel Web (Code Gen) | Web cookie | [link](https://v0.dev) | Paste your session cookie from v0.dev (DevTools → Application → Cookies) | — | | `venice-web` | `ven` | Venice Web (Privacy) | Web cookie | [link](https://venice.ai) | Paste your session cookie from venice.ai (DevTools → Application → Cookies) | — | | `yuanbao-web` | `ybw` | Tencent Yuanbao (Free) | Web cookie | [link](https://yuanbao.tencent.com) | Log in to yuanbao.tencent.com, then paste the full Cookie header (DevTools → Network → any /api request → Request Headers → Cookie). It must contain hy_user and hy_token. | — | -| `zai-web` | `zw` | Z.ai Web (Free) | Web cookie | [link](https://chat.z.ai) | Paste the full Cookie header from chat.z.ai (must include the token= cookie) | — | +| `zai-web` | `zw` | Z.ai Web | Web cookie | [link](https://chat.z.ai) | Copy the "token" value from chat.z.ai → DevTools → Application → Local Storage. Do not copy cookies; OmniRoute handles the per-request CAPTCHA through its browser transport. | — | | `zenmux-free` | `zmf` | ZenMux Free (Web) | Web cookie | [link](https://zenmux.ai) | Login at zenmux.ai, then export all cookies using EditThisCookie or Cookie-Editor and paste the full Cookie header string here. Refresh every ~30 days. | — | -## API Key Providers (paid / paid-with-free-credits) (196) +## API Key Providers (paid / paid-with-free-credits) (227) | ID | Alias | Name | Tags | Website | Notes | |----|-------|------|------|---------|-------| | `360ai` | `360ai` | 360 AI | API key | [link](https://ai.360.cn) | Get API key at ai.360.cn | | `agentrouter` | `agentrouter` | AgentRouter | API key, aggregator | [link](https://agentrouter.org) | $200 free credits on signup - multi-model routing gateway | -| `agnes` | `agnes` | Agnes AI | API key | [link](https://agnes-ai.com) | Get API key at agnes-ai.com | +| `agnes` | `agnes` | Agnes AI | API key, video | [link](https://agnes-ai.com) | Get API key at agnes-ai.com | | `ai21` | `ai21` | AI21 Labs | API key | [link](https://www.ai21.com) | $10 trial credits on signup (valid 3 months), no credit card required | | `aimlapi` | `aiml` | AI/ML API | API key, aggregator | [link](https://aimlapi.com) | Free tier paused (2026) — AI/ML API is now pay-as-you-go only (min $20 top-up); no recurring free credits. | | `ainative` | `ainative` | AINative Studio | API key | [link](https://ainative.studio) | Create a free API key at ainative.studio (no card), then paste it here as a Bearer token. | @@ -112,8 +133,10 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each | `alibaba-cn` | `ali-cn` | Alibaba (China) | API key | [link](https://dashscope.console.aliyun.com/) | — | | `ant-ling` | `ling` | Ant Ling / Ring (inclusionAI) | API key | [link](https://developer.ant-ling.com/en/docs/) | Register and create an API key at the Ant Ling API console (https://chat.ant-ling.com/open), then paste it here. OmniRoute routes chat traffic to https://api.ant-ling.com/v1/chat/completions; the provider is OpenAI-compatible and also exposes an Anthropic-compatible surface. | | `anthropic` | `anthropic` | Anthropic | API key | [link](https://platform.claude.com) | — | +| `anyapi` | `anyapi` | AnyAPI AI | API key, aggregator | [link](https://anyapi.ai) | Free plan: 100,000 ANY Tokens/day and 100 RPM for eligible Free/Basic models; no credit card required. | | `api-airforce` | `af` | Api.airforce | API key | [link](https://api.airforce) | 55 free tier models including Grok-3, Claude 3.7, Qwen3, Kimi-K2, Gemini 2.5 Flash, DeepSeek-V3 | | `arcee-ai` | `arcee` | Arcee AI | API key | [link](https://arcee.ai) | Get API key at arcee.ai | +| `auriko` | `auriko` | Auriko | API key, aggregator | [link](https://www.auriko.ai) | Free plan publishes 1,000 Platform RPM and 10,000 BYOK RPM. Platform inference still passes through provider cost; this is not a free-token pool or unlimited free inference. | | `azure-ai` | `azure-ai` | Azure AI Foundry | API key, enterprise | [link](https://learn.microsoft.com/azure/ai-foundry) | Use your Azure AI Foundry key. Base URL can be https://.services.ai.azure.com/openai/v1/ or https://.openai.azure.com/openai/v1/. | | `azure-openai` | `azure` | Azure OpenAI | API key, enterprise | [link](https://azure.microsoft.com/products/ai-services/openai-service) | Use your Azure OpenAI API key. Base URL should be your resource endpoint, for example https://my-resource.openai.azure.com. | | `bai` | `bai` | b.ai | API key | [link](https://b.ai) | Bearer API key for the b.ai OpenAI-compatible LLM gateway (distinct from TheB.AI). Create a key at https://docs.b.ai, then use https://api.b.ai/v1 as the OpenAI-compatible base URL. | @@ -130,10 +153,13 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each | `bytez` | `bytez` | Bytez | API key | [link](https://bytez.com) | $1 free credits, refreshes every 4 weeks | | `cerebras` | `cerebras` | Cerebras | API key | [link](https://inference.cerebras.ai) | Free Trial: 1M tokens/day, 30K TPM, 5 RPM — no credit card. | | `charm-hyper` | `charm-hyper` | Charm Hyper | API key | [link](https://hyper.charm.land) | 100 free monthly Hypercredits on signup | +| `chat-oripe` | `chat-oripe` | Chat Oripe | API key, aggregator | [link](https://api.oriper.com) | Official metadata advertises 2M tokens/month, but the public site and documentation were blocked during audit; treat the quota and brand mapping as unconfirmed. | +| `chatanywhere` | `chatanywhere` | ChatAnywhere | API key, aggregator | [link](https://chatanywhere.tech) | Personal, educational or research use only: public documentation cites 10,000 points/day and 200 requests/day per IP/key; do not use for commercial traffic. | | `cheaperinference` | `cinf` | Cheaper Inference | API key | [link](https://cheaperinference.com/?utm_source=omniroute) | — | | `chenzk` | `chenzk` | Chenzk API | API key | [link](https://chenzk.top) | — | | `chutes` | `chutes` | Chutes.ai | API key, aggregator | [link](https://chutes.ai) | Bearer API key for the Chutes OpenAI-compatible gateway. | | `clarifai` | `clarifai` | Clarifai | API key, enterprise | [link](https://docs.clarifai.com) | Use your Clarifai PAT or app-specific API key. OmniRoute targets the OpenAI-compatible endpoint at https://api.clarifai.com/v2/ext/openai/v1 and authenticates with Authorization: Key . | +| `cloudcode-one` | `cloudcode-one` | CloudCode.ONE | API key, aggregator | [link](https://cloudcode.one) | Published free models include glm-4.7-flash and glm-4.6v-flash; no numeric quota is published, and key creation may require credit or a coupon. | | `cloudflare-ai` | `cf` | Cloudflare Workers AI | API key | [link](https://developers.cloudflare.com/workers-ai) | Requires API Token AND Account ID (found at dash.cloudflare.com) | | `clova-studio` | `clova` | Naver CLOVA Studio | API key | [link](https://api.ncloud-docs.com/docs/en/ai-naver-clovastudio-summary) | — | | `codestral` | `codestral` | Codestral | API key | [link](https://mistral.ai) | — | @@ -141,9 +167,10 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each | `command-code` | `cmd` | Command Code | API key | [link](https://commandcode.ai/) | Use a Command Code API key. Requests are sent to Command Code's /alpha/generate endpoint. | | `coze` | `coze` | Coze | API key | [link](https://coze.com) | Get API key at coze.com/open/api | | `crof` | `crof` | CrofAI | API key | [link](https://crof.ai) | — | -| `dahl` | `dahl` | Dahl | API key | [link](https://inference.dahl.global) | Click 'Add Account' to auto-generate a token. | +| `dahl` | `dahl` | Dahl | API key | [link](https://inference.dahl.global) | Click 'Add Account' to auto-generate a token, or add a manual API key. | | `databricks` | `databricks` | Databricks | API key, enterprise | [link](https://www.databricks.com) | — | | `datarobot` | `datarobot` | DataRobot | API key, enterprise | [link](https://docs.datarobot.com) | Use your DataRobot API token. Optional Base URL can be the account root (for LLM Gateway) or a deployment URL under /api/v2/deployments/. | +| `deepai` | `deepai` | DeepAI | API key, image | [link](https://deepai.org) | Use your DeepAI API key. Get one at deepai.org — requires a Pro subscription ($9.99/mo). | | `deepinfra` | `deepinfra` | DeepInfra | API key | [link](https://deepinfra.com) | Free signup credits for API testing and model exploration | | `deepseek` | `ds` | DeepSeek | API key | [link](https://platform.deepseek.com) | 5M free tokens on signup - no credit card required | | `dgrid` | `dgrid` | DGrid | API key | [link](https://dgrid.ai) | DGrid Free Models Router: 10 requests/minute and 100 requests/day. A $5 lifetime top-up unlocks up to 20 requests/minute and 1,000 requests/day. | @@ -151,13 +178,18 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each | `digitalocean` | `digitalocean` | DigitalOcean | API key | [link](https://docs.digitalocean.com/products/ai-platform/) | — | | `dit` | `dai` | DIT.ai | API key | [link](https://dit.ai) | Use your dit.ai API key in Authorization: Bearer . Fully OpenAI-compatible — a drop-in replacement, just change the base URL to https://api.dit.ai/v1. | | `doubao` | `doubao` | Doubao | API key | [link](https://doubao.com) | Get API key at console.volcengine.com | +| `dxnt` | `dxnt` | DXNT / DX Token | API key, aggregator | [link](https://www.dxnt.com) | Free accounts are documented at 100 calls/day; the quota may increase through invitations and can vary by account. | +| `electronhub` | `electronhub` | Electron Hub | API key, aggregator | [link](https://www.electronhub.ai) | Free plan: 5 RPM, $0.25 weekly credits and 10 Neutrinos/day for :free models; family budgets also apply. | | `empower` | `empower` | Empower | API key, aggregator | [link](https://docs.empower.dev) | Bearer API key for the Empower OpenAI-compatible endpoint. | | `factory` | `factory` | Factory | API key | [link](https://factory.ai) | Bearer API key for the Factory OpenAI-compatible gateway. | | `fal-ai` | `fal` | Fal.ai | API key, image | [link](https://fal.ai) | — | +| `fastrouter` | `fastrouter` | FastRouter | API key, aggregator | [link](https://fastrouter.ai) | Models with the :free suffix allow 10 requests/day per organization and model; availability may change. | | `featherless-ai` | `featherless` | Featherless AI | API key | [link](https://featherless.ai) | Free tier available — no credit card required | | `fenayai` | `fenayai` | FenayAI | API key, aggregator | [link](https://fenayai.com) | Bearer API key for the FenayAI OpenAI-compatible gateway. | | `fireworks` | `fireworks` | Fireworks AI | API key | [link](https://fireworks.ai) | $1 free starter credits on signup for API testing | +| `free-ai` | `free-ai` | Free.ai | API key, aggregator | [link](https://free.ai) | 30,000 tokens/day cover self-hosted models after email verification. Usage beyond the pool can bill at raw cost, and premium external models are paid. | | `freeaiapikey` | `faik` | FreeAIAPIKey | API key | [link](https://freeaiapikey.com) | — | +| `freeinference` | `freeinference` | FreeInference | API key, aggregator | [link](https://freeinference.org) | Free research access without a card; non-Harvard applicants require manual approval and no numeric quota is publicly guaranteed. | | `freemodel-dev` | `fmd` | FreeModel.dev | API key | [link](https://freemodel.dev) | $300 free credits on signup — no credit card required. Access GPT-5.4 and GPT-5.5 (OpenAI's latest flagship models) through an OpenAI-compatible API. | | `freepik` | `fpk` | Freepik (Mystic) | API key, image | [link](https://freepik.com) | Get API key at freepik.com/developers (Mystic image endpoint) | | `freetheai` | `fta` | FreeTheAi | API key, aggregator | [link](https://freetheai.xyz) | Join the FreeTheAi Discord to get your free API key. | @@ -171,7 +203,6 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each | `gemini` | `gemini` | Gemini (Google AI Studio) | API key | [link](https://aistudio.google.com) | Free forever: 1,500 req/day for Gemini 2.5 Flash — no credit card, get key at aistudio.google.com | | `getgoapi` | `ggo` | GoAPI | API key, aggregator | [link](https://api.getgoapi.com) | — | | `gigachat` | `gigachat` | GigaChat (Sber) | API key | [link](https://developers.sber.ru) | — | -| `github-models` | `ghm` | GitHub Models | API key | [link](https://github.com/marketplace/models) | Create a GitHub PAT with 'models: read' scope at github.com/settings/tokens | | `gitlab` | `gitlab` | GitLab Duo PAT | API key | [link](https://docs.gitlab.com/user/duo_agent_platform/code_suggestions/) | GitLab personal access token for the public Code Suggestions API. Configure a self-hosted base URL when not using gitlab.com. | | `gitlawb` | `glb` | Gitlawb Opengateway (MiMo) | API key | [link](https://opengateway.gitlawb.com) | Free MiMo (xiaomi/mimo-v2.5) revoked 2026-05 — Opengateway is now a pay-as-you-go credit gateway; no recurring free model. | | `gitlawb-gmi` | `glb-gmi` | Gitlawb Opengateway (GMI Cloud) | API key | [link](https://opengateway.gitlawb.com) | Free Nemotron promo ended 2026-06 — the GMI Cloud route is now pay-as-you-go credit only. | @@ -182,6 +213,7 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each | `hackclub` | `hc` | Hackclub AI | API key, aggregator | [link](https://ai.hackclub.com) | Sign in with your Hack Club account at ai.hackclub.com. | | `haiper` | `hp` | Haiper | API key, video | [link](https://haiper.ai) | Get API key at haiper.ai/haiper-api | | `hcnsec` | `hcnsec` | Huancheng Public API | API key | [link](https://api.hcnsec.cn) | Get API key at api.hcnsec.cn | +| `helyxai` | `helyxai` | Helyx AI | API key, aggregator | [link](https://helyxai.space) | Operational Free plan documents 100,000 tokens/day; the site's separate 2M+ marketing claim conflicts and is not treated as a quota guarantee. | | `heroku` | `heroku` | Heroku AI | API key, enterprise | [link](https://www.heroku.com) | — | | `huggingface` | `hf` | HuggingFace | API key | [link](https://huggingface.co) | Free Inference API for thousands of models (Whisper, VITS, SDXL…) | | `hyperbolic` | `hyp` | Hyperbolic | API key | [link](https://hyperbolic.xyz) | $1-5 trial credits on signup for serverless inference | @@ -201,20 +233,29 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each | `laozhang` | `lz` | LaoZhang AI | API key, aggregator | [link](https://api.laozhang.ai) | — | | `leonardo` | `leo` | Leonardo AI | API key, video | [link](https://leonardo.ai) | Get API key at leonardo.ai/developer | | `liquid` | `liquid` | Liquid AI | API key | [link](https://liquid.ai) | Get API key at liquid.ai | +| `literouter` | `literouter` | LiteRouter | API key, aggregator | [link](https://literouter.com) | Free model variants use the :free suffix; daily credit limits vary by model and free input is capped at 5,000 tokens. | | `llamagate` | `llamagate` | LlamaGate | API key | [link](https://llamagate.ai) | — | -| `llm7` | `llm7` | LLM7.io | API key | [link](https://llm7.io) | No signup required - 2 req/s, 20 RPM, 100 req/hr free tier | +| `llm-kiwi` | `llmkiwi` | LLM.Kiwi | API key, aggregator | [link](https://llm.kiwi) | Free plan exposes auto and hrLLM; the published 40 requests/hour limit applies to hrLLM. | +| `llm7` | `llm7` | LLM7.io | API key | [link](https://llm7.io) | Use any non-empty key (for example 'unused'). If older built-in models return model_unavailable, use Available Models → Import from /models or Auto-Sync; verified live model: gemini-3.1-flash-lite. | +| `llmgateway` | `llmgateway` | LLM Gateway | API key, aggregator | [link](https://llmgateway.io) | Hosted Free plan: free-priced models are limited to 5 requests per 10 minutes when the account has no credits. | | `longcat` | `lc` | LongCat AI | API key | [link](https://longcat.chat/platform/docs) | Free: one-time 10M-token grant after account signup + KYC verification (LongCat-2.0). One-time only — not a recurring daily/monthly allowance. | | `maritalk` | `maritalk` | Maritalk | API key | [link](https://www.maritaca.ai) | — | +| `meganova-ai` | `meganova-ai` | MegaNova AI | API key, aggregator | [link](https://meganova.ai) | Free signup without a card. Published Tier 1 per-model quotas total 550 requests/day; they are not a shared global pool, and paid overage can apply if enabled. | | `meta-llama` | `meta` | Meta Llama API | API key | [link](https://llama.developer.meta.com) | — | | `minimax` | `minimax` | Minimax Coding | API key, video | [link](https://www.minimax.io) | — | | `minimax-cn` | `minimax-cn` | Minimax (China) | API key | [link](https://www.minimaxi.com) | — | | `mistral` | `mistral` | Mistral | API key | [link](https://mistral.ai) | Free Experiment tier: rate-limited access to all models, no credit card required | | `mixedbread` | `mxbai` | Mixedbread AI | API key | [link](https://www.mixedbread.com) | Bearer API key for the Mixedbread embeddings API. | +| `mixlayer` | `mixlayer` | Mixlayer | API key, aggregator | [link](https://www.mixlayer.com) | The qwen/qwen3.5-4b-free model is free for prototyping and rate-limited; no fixed public RPM or daily quota is confirmed. | +| `mnn-ai` | `mnn-ai` | MNN AI | API key, aggregator | [link](https://mnnai.ru) | Free plan: $1 monthly credits, 10 RPM and access only to models marked Free. | | `modal` | `mdl` | Modal | API key, enterprise | [link](https://modal.com/docs) | Use the bearer token that protects your Modal deployment, if enabled. Base URL should point to your OpenAI-compatible Modal app, for example https://--.modal.run/v1. | | `modelscope` | `ms` | ModelScope | API key | [link](https://modelscope.cn) | Free tier via ModelScope API-Inference — Alibaba account required. | | `monsterapi` | `monster` | MonsterAPI | API key | [link](https://monsterapi.ai) | Get API key at monsterapi.ai | | `moonshot` | `moonshot` | Kimi | API key | [link](https://platform.kimi.ai?aff=omniroute) | — | | `morph` | `morph` | Morph | API key | [link](https://morphllm.com) | Free tier: 250K credits/month, $0 | +| `muse-code` | `mc` | Muse Code (Meta) | API key | [link](https://github.com/meta-llama/llama-stack) | Use your META_API_KEY env var as a Bearer token. Muse Code CLI uses the OpenAI Responses API wire format (POST /responses). | +| `naga-ac` | `naga` | Naga.ac | API key, aggregator | [link](https://naga.ac) | Get API key at naga.ac — Google/GitHub/Discord signup available. | +| `naga-ai` | `naga-ai` | Naga AI | API key, aggregator | [link](https://naga.ac) | Models marked :free are publicly listed, but no numeric quota is confirmed. Naga's policy warns that free-tier prompts and outputs may be collected or used for training. | | `nanogpt` | `nanogpt` | NanoGPT | API key | [link](https://nano-gpt.com) | — | | `nara` | `nara` | NaraRouter | API key | [link](https://bynara.id) | Get a free API key via NaraRouter's Telegram channel, then paste it here as a Bearer token. | | `navy` | `navy` | NavyAI | API key | [link](https://api.navy) | Create a free API key from the NavyAI dashboard, then paste it here as a Bearer token. | @@ -227,11 +268,13 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each | `nube` | `nube` | Nube.sh | API key | [link](https://nube.sh) | — | | `nvidia` | `nvidia` | NVIDIA NIM | API key | [link](https://build.nvidia.com) | Free dev access: ~40 RPM, 70+ models (Kimi K2.5, GLM 4.7, DeepSeek V3.2...) | | `oci` | `oci` | OCI Generative AI | API key, enterprise | [link](https://www.oracle.com/artificial-intelligence/generative-ai) | Use your OCI Generative AI API key or IAM bearer token. Base URL can be https://inference.generativeai..oci.oraclecloud.com/openai/v1/. | +| `ofoxai` | `ofoxai` | OfoxAI | API key, aggregator | [link](https://ofox.ai) | The current catalog advertises 10+ free models without a public numeric quota; review upstream provenance, retention and training terms before production use. | | `ollama-cloud` | `ollamacloud` | Ollama Cloud | API key | [link](https://ollama.com/settings/keys) | — | | `openadapter` | `oad` | OpenAdapter | API key | [link](https://openadapter.dev) | Use your OpenAdapter API key in Authorization: Bearer sk-cv-. Fully OpenAI-compatible. API base URL: https://api.openadapter.in/v1. | | `openai` | `openai` | OpenAI | API key | [link](https://platform.openai.com) | — | | `opencode-go` | `opencode-go` | OpenCode Go | API key | [link](https://opencode.ai/go) | — | | `opencode-zen` | `opencode-zen` | OpenCode Zen | API key | [link](https://opencode.ai/zen) | — | +| `openference-api` | `ofa` | Openference API | API key | [link](https://openference.com) | Free plan: 3-day trial with open-source models — no credit card required | | `openrouter` | `openrouter` | OpenRouter | API key, aggregator | [link](https://openrouter.ai) | Free models at $0/token with :free suffix - 20 RPM / 200 RPD | | `openvecta` | `openvecta` | OpenVecta | API key | [link](https://openvecta.com) | Free credits on signup for OpenAI-compatible inference across LLMs, embeddings, and reasoning models | | `orcarouter` | `orcarouter` | OrcaRouter | API key | [link](https://www.orcarouter.ai) | — | @@ -241,7 +284,9 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each | `pioneer` | `pn` | Pioneer AI | API key | [link](https://pioneer.ai) | $75 free usage credits — no credit card required | | `plamo` | `plamo` | PLaMo | API key | [link](https://plamo.preferredai.jp/api) | — | | `poe` | `poe` | Poe | API key, aggregator | [link](https://creator.poe.com/api-reference) | Bearer API key for the Poe OpenAI-compatible API. | -| `pollinations` | `pol` | Pollinations AI | API key, video | [link](https://pollinations.ai) | Free keyless tier: openai, openai-fast, openai-large, qwen-coder, mistral, deepseek, grok, gemini-flash-lite-3.1, perplexity-fast, perplexity-reasoning. Premium models (claude, gemini, midijourney) require a Pollinations API key from enter.pollinations.ai. | +| `poixe-ai` | `poixe-ai` | Poixe AI | API key, aggregator | [link](https://poixe.com) | Current public free limits are small and model-group specific: 2 RPM/5 RPD for large-cup models and 20 RPM/50 RPD for small-cup models. | +| `pollinations` | `pol` | Pollinations AI | API key, video | [link](https://pollinations.ai) | Anonymous/keyless access to the documented free models is best-effort. Local v3.8.50 verification (2026-07-31) returned 401 via OmniRoute and Cloudflare 1010 on direct upstream probes from the same network. Premium models still require a Pollinations API key from enter.pollinations.ai. | +| `poolside` | `poolside` | Poolside | API key | [link](https://poolside.ai) | Laguna S 2.1 and XS 2.1 are free during Preview; no public numeric quota is published. | | `predibase` | `predibase` | Predibase | API key | [link](https://predibase.com) | ⚠️ **DEPRECATED.** serving.app.predibase.com no longer resolves (sweep 2026-06-19); the managed serving API appears discontinued. | | `publicai` | `publicai` | PublicAI | API key | [link](https://publicai.co) | Requires an API key — one-time signup credit, then paid | | `puter` | `pu` | Puter AI | API key | [link](https://puter.com) | Get token at puter.com/dashboard → Copy Auth Token | @@ -250,6 +295,7 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each | `qwen-cloud` | `qwc` | Qwen Cloud | API key | [link](https://www.qwencloud.com/) | — | | `qwen-cloud-token-plan` | `qct` | Qwen Cloud Token Plan | API key | [link](https://www.qwencloud.com/pricing/token-plan) | — | | `recraft` | `recraft` | Recraft | API key, image | [link](https://recraft.ai) | — | +| `regolo` | `regolo` | Regolo AI | API key | [link](https://regolo.ai) | Get your Regolo API key from regolo.ai, then paste it here as a Bearer token. | | `reka` | `reka` | Reka | API key | [link](https://docs.reka.ai/chat/overview) | Use your Reka API key. OmniRoute supports the OpenAI-compatible base URL https://api.reka.ai/v1 and sends both Authorization and X-Api-Key headers for compatibility. | | `requesty` | `requesty` | Requesty | API key | [link](https://requesty.ai) | Free tier ~200 requests/day - multi-model routing gateway (300+ models) | | `routeway` | `routeway` | Routeway | API key | [link](https://routeway.ai) | Create a free API key at routeway.ai, then paste it here as a Bearer token. | @@ -264,6 +310,7 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each | `siliconflow` | `siliconflow` | SiliconFlow | API key | [link](https://cloud.siliconflow.com) | $1 free credits plus permanently free models after identity verification | | `snowflake` | `snowflake` | Snowflake Cortex | API key, enterprise | [link](https://www.snowflake.com) | — | | `sparkdesk` | `sparkdesk` | SparkDesk | API key | [link](https://xinghuo.xfyun.cn) | Get API key at console.xfyun.cn | +| `speka` | `speka` | Speka AI | API key, aggregator | [link](https://speka.me) | Free plan: $1 monthly usage, 10 RPM, one API key and access to open models and the playground; no card required. | | `stability-ai` | `stability` | Stability AI | API key, image | [link](https://stability.ai) | — | | `stepfun` | `stepfun` | StepFun | API key | [link](https://stepfun.com) | Get API key at platform.stepfun.com | | `sumopod` | `sumopod` | SumoPod | API key | [link](https://ai.sumopod.com) | Use your SumoPod API key (sk-...) in Authorization: Bearer . Fully OpenAI-compatible. API base URL: https://ai.sumopod.com/v1. | @@ -273,11 +320,13 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each | `thebai` | `thebai` | TheB.AI | API key, aggregator | [link](https://theb.ai) | Bearer API key for the TheB.AI OpenAI-compatible gateway. | | `tinyfish` | `tf` | TinyFish Fetch | API key | [link](https://docs.tinyfish.ai/fetch-api) | X-API-Key from agent.tinyfish.ai/api-keys | | `together` | `together` | Together AI | API key, video | [link](https://www.together.ai) | — | +| `tokenreply` | `tokenreply` | TokenReply | API key, aggregator | [link](https://www.tokenreply.com) | Free-tagged models have model- and campaign-specific daily limits; no fixed global free quota is published. | | `tokenrouter` | `trk` | TokenRouter | API key | [link](https://tokenrouter.com) | Use your TokenRouter API key in Authorization: Bearer . Fully OpenAI-compatible. API base URL: https://api.tokenrouter.com/v1. | | `topaz` | `topaz` | Topaz | API key, image | [link](https://topazlabs.com) | — | | `typhoon` | `typhoon` | Typhoon | API key | [link](https://docs.opentyphoon.ai) | Free API key with a 5 req/s and 200 req/m rate limit. | | `udio` | `udio` | Udio | API key | [link](https://udio.com) | Paste session cookie from udio.com (Supabase auth) | -| `uncloseai` | `unc` | UncloseAI | API key | [link](https://uncloseai.com) | No auth required. API accepts any non-empty string as key for identification. | +| `uncloseai` | `unc` | UncloseAI | API key | [link](https://uncloseai.com) | No auth required. API accepts any non-empty string as key for identification. If older built-in models return 404, use Available Models → Import from /models or Auto-Sync; verified live model: solidrust/Hermes-3-Llama-3.1-8B-AWQ. | +| `unorouter` | `unorouter` | UnoRouter | API key, aggregator | [link](https://unorouter.ai) | Models with the :free suffix do not debit balance; limit is 1 request/minute per free model per user. | | `upstage` | `upstage` | Upstage | API key | [link](https://www.upstage.ai) | — | | `v0-vercel` | `v0` | v0 (Vercel) | API key | [link](https://v0.dev) | — | | `venice` | `venice` | Venice.ai | API key | [link](https://venice.ai) | — | @@ -295,8 +344,11 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each | `xiaomi-mimo` | `mimo` | Xiaomi MiMo | API key | [link](https://mimo.mi.com) | — | | `xiaomi-mimo-token-plan` | `mimotp` | Xiaomi MiMo Token Plan | API key | [link](https://mimo.mi.com) | — | | `yi` | `yi` | Yi (01.AI) | API key | [link](https://01.ai) | Get API key at platform.lingyiwanwu.com | +| `yolo-auto` | `yolo-auto` | Yolo-Auto | API key, aggregator | [link](https://yolo-auto.com) | Free API access is request-limited and intended for testing; no numeric daily quota is published and free access is not promised indefinitely. | | `zai` | `zai` | Z.AI | API key | [link](https://open.bigmodel.cn) | — | | `zenmux` | `zm` | ZenMux | API key | [link](https://zenmux.ai) | Use your ZenMux API key in Authorization: Bearer . ZenMux is fully OpenAI-compatible. Base URL: https://zenmux.ai/api/v1. | +| `zerolimitai` | `zerolimitai` | ZeroLimitAI | API key, aggregator | [link](https://www.zerolimitai.com) | Temporary free trial is advertised, but official pages conflict between 3 and 7 days; a 100-calls/day claim is not treated as permanent. | +| `zylo-api` | `zylo` | Zylo API | API key, aggregator | [link](https://zyloai.net) | Basic plan: 10 RPM, 7,200 requests/day and 200,000 tokens/day; limited to Basic text models. | ## Local Providers (12) @@ -321,7 +373,7 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each |----|-------|------|------|---------|-------| | `brave-search` | `brave-search` | Brave Search | Search | [link](https://brave.com/search/api) | Subscription token from Brave Search API dashboard | | `exa-search` | `exa-search` | Exa Search | Search | [link](https://exa.ai) | API key from dashboard.exa.ai | -| `firecrawl` | `fc` | Firecrawl | Search | [link](https://firecrawl.dev) | — | +| `firecrawl` | `fc` | Firecrawl | Search | [link](https://firecrawl.dev) | API key from firecrawl.dev/app/api-keys (or set your self-hosted Firecrawl base URL) | | `google-pse-search` | `google-pse` | Google Programmable Search | Search | [link](https://developers.google.com/custom-search/v1/overview) | Requires a Google API key and your Programmable Search Engine ID (cx) | | `linkup-search` | `linkup` | Linkup Search | Search | [link](https://docs.linkup.so) | Bearer API key from the Linkup dashboard | | `ollama-search` | `ollama-search` | Ollama Search | Search | [link](https://ollama.com/settings/keys) | Same API key as Ollama Cloud (from ollama.com/settings/keys) | @@ -332,7 +384,7 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each | `tavily-search` | `tavily-search` | Tavily Search | Search | [link](https://tavily.com) | API key from app.tavily.com (format: tvly-...) | | `youcom-search` | `youcom-search` | You.com Search | Search | [link](https://you.com/business/api/) | X-API-Key from the You.com platform dashboard | -## Audio-only Providers (11) +## Audio-only Providers (12) | ID | Alias | Name | Tags | Website | Notes | |----|-------|------|------|---------|-------| @@ -346,6 +398,7 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each | `inworld` | `inworld` | Inworld | Audio | [link](https://inworld.ai) | — | | `playht` | `playht` | PlayHT | Audio | [link](https://play.ht) | — | | `rev-ai` | `revai` | Rev AI | Audio | [link](https://www.rev.ai) | — | +| `soniox` | `sx` | Soniox | Audio | [link](https://soniox.com) | — | | `speechmatics` | `sm` | Speechmatics | Audio | [link](https://www.speechmatics.com) | Free tier — 8 hours/month, no credit card required. Batch (async) mode only. | ## Upstream Proxy Providers (2) @@ -373,7 +426,7 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each - Catalog: [`src/shared/constants/providers.ts`](../../src/shared/constants/providers.ts) - Registry (per-model details): [`open-sse/config/providerRegistry.ts`](../../open-sse/config/providerRegistry.ts) -- Executors: [`open-sse/executors/`](../../open-sse/executors/) (31 files) +- Executors: [`open-sse/executors/`](../../open-sse/executors/) (101 implementations) - Translators: [`open-sse/translator/`](../../open-sse/translator/) ## See Also diff --git a/docs/routing/AUTO-COMBO.md b/docs/routing/AUTO-COMBO.md index 1b42354be7..7a0b387191 100644 --- a/docs/routing/AUTO-COMBO.md +++ b/docs/routing/AUTO-COMBO.md @@ -161,11 +161,11 @@ The detection helper lives in `src/lib/combos/modelNameCollision.ts`. ## How It Works (Persisted Auto-Combos) -The Auto-Combo Engine dynamically selects the best provider/model for each request using a **13-factor scoring function** (defined in `open-sse/services/autoCombo/scoring.ts` → `DEFAULT_WEIGHTS`). All weights sum to **1.0**. +The Auto-Combo Engine dynamically selects the best provider/model for each request using a **14-factor scoring function** (defined in `open-sse/services/autoCombo/scoring.ts` → `DEFAULT_WEIGHTS`). Weights form a normalized distribution (custom weights are renormalized by `normalizeScoringWeights()`). -![Auto-Combo 12-factor scoring](../diagrams/exported/auto-combo-12factor.svg) +![Auto-Combo 14-factor scoring](../diagrams/exported/auto-combo-12factor.svg) -> Source: [diagrams/auto-combo-12factor.mmd](../diagrams/auto-combo-12factor.mmd) (regenerate via `npm run docs:render-diagrams`). Diagram/filename predate the `cacheAffinity` factor added by #8008 and still show 12 factors. +> Source: [diagrams/auto-combo-12factor.mmd](../diagrams/auto-combo-12factor.mmd) (regenerate via `npm run docs:render-diagrams`). The filename predates the current factor set; the diagram shows 13 of the 14 factors (missing `sessionAvailability`). | Factor | Default Weight | Description | | :-------------------- | :------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | @@ -179,11 +179,12 @@ The Auto-Combo Engine dynamically selects the best provider/model for each reque | `tierAffinity` | 0.05 | Affinity between the candidate's tier and the manifest-recommended tier | | `specificityMatch` | 0.05 | Match between request specificity (manifest hint) and model tier | | `contextAffinity` | 0.05 | Affinity between the request's context-window need and the model's context window | +| `sessionAvailability` | 0.05 | OAuth session availability of the candidate connection for this session (`getOAuthSessionAvailability()`; non-OAuth connections score 1.0) | | `connectionDensity` | 0.05 | Spreads load across connections of the same provider (anti-concentration) | | `cacheAffinity` | 0.00 | Rendezvous-hash affinity toward the connection likeliest to already hold this request's prompt-cache prefix (`open-sse/services/combo/promptCacheAffinity.ts`); disabled by default (#8008) | | `resetWindowAffinity` | 0.00 | Bias toward connections whose quota reset window is favorable (disabled by default) | -**Sum:** `0.20 + 0.15 + 0.15 + 0.12 + 0.08 + 0.05 + 0.05 + 0.05 + 0.05 + 0.05 + 0.05 + 0.00 + 0.00 = 1.0` (validated by `validateWeights()`). +**Sum:** `0.20 + 0.15 + 0.15 + 0.12 + 0.08 + 0.05 + 0.05 + 0.05 + 0.05 + 0.05 + 0.05 + 0.05 + 0.00 + 0.00 = 1.05` as literally declared in `DEFAULT_WEIGHTS`; user-configured weights are renormalized into a distribution by `normalizeScoringWeights()` before scoring. ## Mode Packs @@ -654,7 +655,7 @@ Including the bare `auto` (default) plus the 6 `AutoVariant` values declared in ## How tiers fit Auto-Combo -The 12-factor scoring function (`open-sse/services/autoCombo/scoring.ts`) treats tier +The 14-factor scoring function (`open-sse/services/autoCombo/scoring.ts`) treats tier membership as two signals: `tierPriority` (0.05) and `tierAffinity` (0.05). See the canonical [scoring factor table](#how-it-works-persisted-auto-combos) above for the full `DEFAULT_WEIGHTS` set — the per-pack overrides (ship-fast/cost-saver/quality-first/ diff --git a/llm.txt b/llm.txt index daed1b645f..470ad5f415 100644 --- a/llm.txt +++ b/llm.txt @@ -1,6 +1,6 @@ # OmniRoute -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -8,13 +8,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -22,7 +22,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -41,7 +41,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -94,7 +94,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -165,7 +165,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -182,7 +182,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -208,11 +208,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -224,7 +224,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -263,8 +263,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -275,15 +275,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -312,7 +312,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -342,13 +342,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -369,7 +369,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -381,7 +381,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -435,7 +435,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -443,7 +443,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -476,10 +476,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/package.json b/package.json index 86008e9677..e89911492c 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "omniroute", "version": "3.8.50", - "description": "Unified AI router with 291 providers, RTK+Caveman compression, auto fallback, MCP/A2A, desktop, PWA, and OpenAI-compatible APIs.", + "description": "Unified AI router with 338 providers, RTK+Caveman compression, auto fallback, MCP/A2A, desktop, PWA, and OpenAI-compatible APIs.", "type": "module", "bin": { "omniroute": "bin/omniroute.mjs", diff --git a/scripts/check/check-docs-counts-sync.mjs b/scripts/check/check-docs-counts-sync.mjs index c0e4cf83f6..61077b8cf9 100644 --- a/scripts/check/check-docs-counts-sync.mjs +++ b/scripts/check/check-docs-counts-sync.mjs @@ -18,9 +18,13 @@ // Exits 0 on success, 1 on STRICT drift (or any drift with --strict). // Run: node scripts/check/check-docs-counts-sync.mjs // -// NOTE: the provider check trusts PROVIDER_REFERENCE.md as the canonical total. If a -// provider is added to the code but the reference is not regenerated, this guard will -// not catch it — regenerate with `npm run gen:provider-reference` before relying on it. +// NOTE: PROVIDER_REFERENCE.md is no longer blindly trusted — a STRICT check compares +// the doc's `Total providers` against the live provider modules (the same collections +// the generator reads), so a hand-stale doc is a red, not a silently propagated total. +// Fix by running `npm run gen:provider-reference`. Additional STRICT coverage added in +// the 2026-08-12 hardening: llm.txt + package.json description (providers), migration +// count (README/AGENTS/llm.txt), and canonical numbers inside the README SVG diagrams +// (providers / MCP tools / routing strategies / free-tier pools). import fs from "node:fs"; import { spawnSync } from "node:child_process"; @@ -76,6 +80,13 @@ export function readProviderTotal() { return parseProviderTotal(fs.readFileSync(abs, "utf8")); } +// STRICT: number of SQL migration files shipped with the app. +export function countMigrations() { + const abs = path.join(ROOT, "src", "lib", "db", "migrations"); + if (!fs.existsSync(abs)) return 0; + return fs.readdirSync(abs).filter((f) => f.endsWith(".sql")).length; +} + // STRICT: canonical i18n locale count, read from the shared config. export function countLocales() { const abs = path.join(ROOT, "config", "i18n.json"); @@ -148,6 +159,15 @@ function readCodeFacts() { 'import {notionTools} from "./open-sse/mcp-server/tools/notionTools.ts";', 'import {obsidianTools} from "./open-sse/mcp-server/tools/obsidianTools.ts";', 'import {compressionTools} from "./open-sse/mcp-server/tools/compressionTools.ts";', + // Live provider total — the SAME collections gen-provider-reference.ts unions, so the + // doc-vs-live check below cannot drift from the generator's definition of "provider". + 'import * as PROV from "./src/shared/constants/providers.ts";', + "const provCols=[PROV.FREE_PROVIDERS,PROV.NOAUTH_PROVIDERS,PROV.OAUTH_PROVIDERS,", + "PROV.WEB_COOKIE_PROVIDERS,PROV.APIKEY_PROVIDERS,PROV.LOCAL_PROVIDERS,PROV.SEARCH_PROVIDERS,", + "PROV.AUDIO_ONLY_PROVIDERS,PROV.UPSTREAM_PROXY_PROVIDERS,PROV.CLOUD_AGENT_PROVIDERS,", + "PROV.SYSTEM_PROVIDERS];", + "const pids=new Set();", + "for(const c of provCols)for(const p of Object.values(c||{}))if(p&&p.id)pids.add(p.id);", "const cols={MCP_TOOLS,memoryTools,skillTools,agentSkillTools,githubSkillTools,poolTools,", "gamificationTools,pluginTools,notionTools,obsidianTools,compressionTools};", "const sc=new Set();", @@ -158,7 +178,7 @@ function readCodeFacts() { 'console.log("@@"+JSON.stringify({freeSteady:t.steadyRecurringTokens,', "freeFirst:t.firstMonthRealisticTokens,freePools:t.poolCount,engines:ENGINE_IDS.length,", "cliTotal:cli.length,cliCode:by('code'),cliAgent:by('agent'),", - "mcpTools:countUniqueMcpTools(cols),mcpScopes:sc.size}));", + "mcpTools:countUniqueMcpTools(cols),mcpScopes:sc.size,providers:pids.size}));", ].join(""); const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "docs-counts-")); try { @@ -252,6 +272,82 @@ export function makeNumberClaimValidator(expected, opts) { }; } +// --- v3.8.50 hardening validators -------------------------------------------- +// PURE: doc total must equal the live provider-module total (closes the falso-verde +// found in the 2026-08-12 audit: the doc sat hand-stale at 291 while the modules +// defined 338, and every downstream check inherited the stale total). +export function makeProviderReferenceValidator(expected) { + return (content) => { + const total = parseProviderTotal(content); + if (!total) return { ok: false, detail: "no `Total providers: **N**` marker found" }; + if (total === expected) + return { ok: true, detail: `doc total ${total} matches the live provider modules` }; + return { + ok: false, + detail: + `doc total ${total} is stale — the live provider modules define ${expected} ` + + `(run npm run gen:provider-reference)`, + }; + }; +} + +// PURE: the npm package description must carry the live provider count. +export function makePackageDescriptionValidator(expected) { + return (content) => { + let desc = ""; + try { + desc = String(JSON.parse(content).description || ""); + } catch { + return { ok: false, detail: "package.json could not be parsed" }; + } + if (desc.includes(String(expected))) + return { ok: true, detail: `description mentions the live provider count ${expected}` }; + return { + ok: false, + detail: `description does not mention the live provider count ${expected}: "${desc}"`, + }; + }; +} + +// PURE: sweep an SVG's text/aria content for the canonical numbers. Patterns are +// deliberately narrow — they anchor on the surrounding words so path coordinates, +// width/font-size attributes and small unrelated counts ("15 providers ToS-flagged", +// "100+ providers") can never register as claims. Providers require 3+ digits for the +// same reason. +const SVG_CANONICAL_PATTERNS = [ + { key: "providers", what: "providers", pattern: /(\d{3,4}) (?:AI )?providers\b/g }, + { key: "mcpTools", what: "MCP tools", pattern: /MCP (?:server with |with |\()(\d+)/g }, + { key: "strategies", what: "routing strategies", pattern: /(\d+) routing strategies\b/g }, + { key: "pools", what: "free-tier pools", pattern: /(\d+) provider pools\b/g }, +]; + +export function checkSvgCanonicalNumbers(content, expected) { + const stale = []; + let claims = 0; + for (const { key, what, pattern } of SVG_CANONICAL_PATTERNS) { + if (expected[key] == null) continue; + for (const m of content.matchAll(pattern)) { + claims++; + const value = Number(m[1]); + if (value !== expected[key]) stale.push(`"${m[0]}" (${what} — code has ${expected[key]})`); + } + } + if (!claims) return { ok: true, detail: "no canonical-number claims in this SVG" }; + if (!stale.length) return { ok: true, detail: `${claims} canonical claim(s) match the code` }; + return { ok: false, detail: `stale: ${[...new Set(stale)].join(", ")}` }; +} + +// The README-embedded diagrams that historically rotted because no gate read them +// (the alt-text in README.md is checked, the SVG text nodes never were). +const SVG_DIAGRAM_FILES = [ + "docs/diagrams/readme-hero.svg", + "docs/diagrams/free-tier-budget.svg", + "docs/diagrams/promise-pillars.svg", + "docs/diagrams/comparison-table.svg", + "docs/diagrams/cli-terminal.svg", + "docs/diagrams/tier-cascade.svg", +]; + export function buildChecks() { return [ { @@ -259,7 +355,26 @@ export function buildChecks() { actual: readProviderTotal(), docKey: "providers", strict: true, - files: ["README.md", "AGENTS.md"], + files: ["README.md", "AGENTS.md", "llm.txt"], + }, + { + label: "Provider count (package.json description)", + actual: readProviderTotal(), + docKey: "providers", + strict: true, + files: ["package.json"], + validate: makePackageDescriptionValidator(readProviderTotal()), + }, + { + label: "DB migrations count", + actual: countMigrations(), + docKey: "migrations", + strict: true, + files: ["README.md", "AGENTS.md", "llm.txt"], + validate: makeNumberClaimValidator(countMigrations(), { + what: "migrations", + pattern: /(\d+)\+? migrations?\b/gi, + }), }, { label: "i18n locales count", @@ -289,6 +404,30 @@ export function buildChecks() { validate: makeNumberClaimValidator(expected, { what, ...opts }), }); return [ + { + label: "Provider reference total (doc vs live modules)", + actual: f.providers, + docKey: "providers (live)", + strict: true, + files: ["docs/reference/PROVIDER_REFERENCE.md"], + validate: makeProviderReferenceValidator(f.providers), + }, + { + label: "SVG canonical numbers (live code)", + actual: + `${f.providers} providers / ${f.mcpTools} MCP tools / ` + + `${countRoutingStrategies()} strategies / ${f.freePools} pools`, + docKey: "SVG canonical numbers", + strict: true, + files: SVG_DIAGRAM_FILES, + validate: (content) => + checkSvgCanonicalNumbers(content, { + providers: f.providers, + mcpTools: f.mcpTools, + strategies: countRoutingStrategies(), + pools: f.freePools, + }), + }, { label: "Free-tier headline (live catalog)", actual: `~${(f.freeSteady / 1e9).toFixed(2)}B steady / ${f.freePools} pools`, diff --git a/scripts/docs/gen-provider-reference.ts b/scripts/docs/gen-provider-reference.ts index 118a7f9bc1..a91ddddbce 100644 --- a/scripts/docs/gen-provider-reference.ts +++ b/scripts/docs/gen-provider-reference.ts @@ -7,6 +7,7 @@ import path from "node:path"; import { fileURLToPath } from "node:url"; import { FREE_PROVIDERS, + NOAUTH_PROVIDERS, OAUTH_PROVIDERS, WEB_COOKIE_PROVIDERS, APIKEY_PROVIDERS, @@ -136,6 +137,7 @@ function buildHeader(total: number): string { "## Categories", "", "- **Free** — free tier with API key (configured via dashboard)", + "- **No-auth** — public endpoints that require no key or sign-in at all", "- **OAuth** — sign-in flow handled by OmniRoute, no API key needed", "- **Web cookie** — wraps the provider's web app via cookie auth", "- **API key** — paid provider configured via API key (free credits may apply)", @@ -159,8 +161,19 @@ function buildHeader(total: number): string { ].join("\n"); } +function countExecutorImpls(): number { + const dir = path.join(ROOT, "open-sse", "executors"); + const nonImpl = new Set(["index.ts", "index.mts", "types.ts", "base.ts", "constants.ts"]); + return fs + .readdirSync(dir) + .filter( + (f) => f.endsWith(".ts") && !f.endsWith(".test.ts") && !f.startsWith("__") && !nonImpl.has(f) + ).length; +} + function main() { const free = asRecords(FREE_PROVIDERS); + const noauth = asRecords(NOAUTH_PROVIDERS as Record); const oauth = asRecords(OAUTH_PROVIDERS); const webCookie = asRecords(WEB_COOKIE_PROVIDERS); const apiKey = asRecords(APIKEY_PROVIDERS); @@ -173,6 +186,7 @@ function main() { const allIds = new Set([ ...free.map((p) => p.id), + ...noauth.map((p) => p.id), ...oauth.map((p) => p.id), ...webCookie.map((p) => p.id), ...apiKey.map((p) => p.id), @@ -186,6 +200,7 @@ function main() { const sections = [ buildSection("Free Tier (OAuth-first or no-key)", free, "Free"), + buildSection("No-auth Providers (no key required)", noauth, "No-auth"), buildSection("OAuth Providers", oauth, "OAuth"), buildSection("Web Cookie Providers", webCookie, "Web cookie"), buildSection("API Key Providers (paid / paid-with-free-credits)", apiKey, "API key"), @@ -202,7 +217,7 @@ function main() { "", "- Catalog: [`src/shared/constants/providers.ts`](../../src/shared/constants/providers.ts)", "- Registry (per-model details): [`open-sse/config/providerRegistry.ts`](../../open-sse/config/providerRegistry.ts)", - "- Executors: [`open-sse/executors/`](../../open-sse/executors/) (31 files)", + `- Executors: [\`open-sse/executors/\`](../../open-sse/executors/) (${countExecutorImpls()} implementations)`, "- Translators: [`open-sse/translator/`](../../open-sse/translator/)", "", "## See Also", @@ -218,9 +233,10 @@ function main() { console.log(`✓ Wrote ${OUT_FILE}`); console.log(` Providers: ${allIds.size} unique IDs`); console.log( - ` Sections: free=${free.length}, oauth=${oauth.length}, web=${webCookie.length}, ` + - `apikey=${apiKey.length}, local=${local.length}, search=${search.length}, ` + - `audio=${audio.length}, proxy=${upstreamProxy.length}, cloud=${cloudAgent.length}, system=${system.length}` + ` Sections: free=${free.length}, noauth=${noauth.length}, oauth=${oauth.length}, ` + + `web=${webCookie.length}, apikey=${apiKey.length}, local=${local.length}, ` + + `search=${search.length}, audio=${audio.length}, proxy=${upstreamProxy.length}, ` + + `cloud=${cloudAgent.length}, system=${system.length}` ); } diff --git a/tests/unit/check-docs-counts-sync.test.ts b/tests/unit/check-docs-counts-sync.test.ts index 27deae3057..9e2afca449 100644 --- a/tests/unit/check-docs-counts-sync.test.ts +++ b/tests/unit/check-docs-counts-sync.test.ts @@ -196,3 +196,87 @@ test("compression-engines and CLI-tools gates catch their v3.8.49 drift", () => assert.equal(cli("all 33 tools (25 CLI Code's)").ok, true); assert.equal(cli("all 26 tools (25 CLI Code's)").ok, false); }); + +// --- v3.8.50 hardening: live provider source, llm.txt/package.json, migrations, SVGs -- +// Regression guards for the 2026-08-12 audit: PROVIDER_REFERENCE.md was hand-stale at +// 291 while the live provider modules defined 338, and the gate trusted the doc — so +// README/AGENTS validated against a stale total and the gate stayed falsely green. +import { + countMigrations, + makeProviderReferenceValidator, + makePackageDescriptionValidator, + checkSvgCanonicalNumbers, +} from "../../scripts/check/check-docs-counts-sync.mjs"; + +const countMigs = countMigrations as () => number; +const makeRefValidator = makeProviderReferenceValidator as ( + expected: number +) => (content: string) => { ok: boolean; detail: string }; +const makePkgValidator = makePackageDescriptionValidator as ( + expected: number +) => (content: string) => { ok: boolean; detail: string }; +const checkSvg = checkSvgCanonicalNumbers as ( + content: string, + expected: { providers?: number; mcpTools?: number; strategies?: number; pools?: number } +) => { ok: boolean; detail: string }; + +test("countMigrations reads a real, positive migration count", () => { + assert.ok(countMigs() > 100, "migrations dir should hold > 100 .sql files"); +}); + +test("provider-reference validator accepts the live total and rejects a stale doc", () => { + const v = makeRefValidator(338); + assert.equal(v("Total providers: **338**. See category breakdown below.").ok, true); + const stale = v("Total providers: **291**. See category breakdown below."); + assert.equal(stale.ok, false, "a hand-stale doc total must be a red, not a silent pass"); + assert.match(stale.detail, /gen:provider-reference/); + assert.equal(v("# Provider Reference\n\nNo total marker.").ok, false); +}); + +test("package.json description validator catches a stale provider count", () => { + const v = makePkgValidator(338); + assert.equal(v(JSON.stringify({ description: "Unified AI router with 338 providers" })).ok, true); + assert.equal( + v(JSON.stringify({ description: "Unified AI router with 291 providers" })).ok, + false + ); + assert.equal(v("not json at all {").ok, false); +}); + +test("migrations claim validator accepts the real count and rejects stale styles", () => { + const v = makeValidator(144, { what: "migrations", pattern: /(\d+)\+? migrations?\b/gi }); + assert.equal(v("SQLite domain modules (144 migrations)").ok, true); + assert.equal(v("local, zero-config, 110+ migrations").ok, false); + assert.equal(v("(130 migrations)").ok, false); +}); + +const SVG_EXPECTED = { providers: 338, mcpTools: 105, strategies: 19, pools: 42 }; + +test("SVG gate accepts canonical numbers in text and aria-label claims", () => { + const good = + 'aria-label="338 AI providers, 19 routing strategies, MCP with 105 tools, ' + + '42 provider pools" 338 providersMCP (105'; + assert.equal(checkSvg(good, SVG_EXPECTED).ok, true); +}); + +test("SVG gate flags each stale canonical number the audit found", () => { + for (const stale of [ + "290 providers", + 'aria-label="MCP server with 104 tools"', + "MCP (104", + "18 routing strategies", + 'aria-label="43 provider pools and 460+ models"', + ]) { + const r = checkSvg(stale, SVG_EXPECTED); + assert.equal(r.ok, false, `expected stale claim to be flagged: ${stale}`); + } +}); + +test("SVG gate ignores coordinates, sizes and unrelated small counts", () => { + const noise = + '' + + '15 providers ToS-flagged' + + "100+ providers90+ free"; + const r = checkSvg(noise, SVG_EXPECTED); + assert.equal(r.ok, true, `coordinates/attrs must never register claims: ${r.detail}`); +});