diff --git a/AGENTS.md b/AGENTS.md index acce7a2109..4c9bb63b20 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -46,7 +46,7 @@ Repository map and Reference Documentation sections below. ## Project at a Glance -**OmniRoute** — unified AI proxy/router. One endpoint, 291 LLM providers, auto-fallback. +**OmniRoute** — unified AI proxy/router. One endpoint, 338 LLM providers, auto-fallback. | Layer | Location | Purpose | | ------------- | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | @@ -56,9 +56,9 @@ Repository map and Reference Documentation sections below. | Translators | `open-sse/translator/` | Format conversion (OpenAI↔Claude↔Gemini) | | Transformer | `open-sse/transformer/` | Responses API ↔ Chat Completions | | Services | `open-sse/services/` | Combo routing, rate limits, caching, etc | -| Database | `src/lib/db/` | SQLite domain modules (130 migrations) | +| Database | `src/lib/db/` | SQLite domain modules (144 migrations) | | Domain/Policy | `src/domain/` | Policy engine, cost rules, fallback logic | -| MCP Server | `open-sse/mcp-server/` | 105 tools (42 base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules), 3 transports (stdio / SSE / Streamable HTTP), 31 scopes | +| MCP Server | `open-sse/mcp-server/` | 105 tools (43 base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules), 3 transports (stdio / SSE / Streamable HTTP), 31 scopes | | A2A Server | `src/lib/a2a/` | JSON-RPC 2.0 agent protocol | | Skills | `src/lib/skills/` | Extensible skill framework | | Memory | `src/lib/memory/` | Persistent conversational memory | @@ -83,7 +83,7 @@ Client → /v1/chat/completions (Next.js route) API routes follow a consistent pattern: `Route → CORS preflight → Zod body validation → Optional auth (extractApiKey/isValidApiKey) → API key policy enforcement → Handler delegation (open-sse)`. No global Next.js middleware — interception is route-specific. -**Combo routing** (`open-sse/services/combo.ts`): 19 public strategies (priority, weighted, fill-first, round-robin, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline). Each target calls `handleSingleModel()` which wraps `handleChatCore()` with per-target error handling and circuit breaker checks. The `fusion` strategy is the exception: it fans out to a panel of models in parallel, then a judge model synthesizes one final answer (`open-sse/services/fusion.ts`). See `docs/routing/AUTO-COMBO.md` for the 13-factor Auto-Combo scoring + the full strategy table and `docs/architecture/RESILIENCE_GUIDE.md` for the 3 resilience layers. +**Combo routing** (`open-sse/services/combo.ts`): 19 public strategies (priority, weighted, fill-first, round-robin, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline). Each target calls `handleSingleModel()` which wraps `handleChatCore()` with per-target error handling and circuit breaker checks. The `fusion` strategy is the exception: it fans out to a panel of models in parallel, then a judge model synthesizes one final answer (`open-sse/services/fusion.ts`). See `docs/routing/AUTO-COMBO.md` for the 14-factor Auto-Combo scoring + the full strategy table and `docs/architecture/RESILIENCE_GUIDE.md` for the 3 resilience layers. --- @@ -395,7 +395,7 @@ For any non-trivial change, read the matching deep-dive first: | Repo navigation | `docs/architecture/REPOSITORY_MAP.md` | | Architecture | `docs/architecture/ARCHITECTURE.md` | | Engineering reference | `docs/architecture/CODEBASE_DOCUMENTATION.md` | -| Auto-Combo (13-factor scoring, 19 strategies) | `docs/routing/AUTO-COMBO.md` | +| Auto-Combo (14-factor scoring, 19 strategies) | `docs/routing/AUTO-COMBO.md` | | Resilience (3 mechanisms) | `docs/architecture/RESILIENCE_GUIDE.md` | | Reasoning replay | `docs/routing/REASONING_REPLAY.md` | | Skills framework | `docs/frameworks/SKILLS.md` | @@ -419,7 +419,7 @@ For any non-trivial change, read the matching deep-dive first: | Electron desktop app | `docs/guides/ELECTRON_GUIDE.md` | | Release flow | `docs/ops/RELEASE_CHECKLIST.md` | | Embedded services | `docs/frameworks/EMBEDDED-SERVICES.md` | -| Quality gates (~48 scripts, allowlist policy) | `docs/architecture/QUALITY_GATES.md` | +| Quality gates (~80 scripts, allowlist policy) | `docs/architecture/QUALITY_GATES.md` | --- @@ -611,7 +611,7 @@ focused checks, and use a Conventional Commit message (for example, `docs: slim ## Quality Gates & Ratchets -OmniRoute has **~48 quality-gate scripts** (`scripts/check/` + `scripts/quality/`) wired +OmniRoute has **~80 quality-gate scripts** (`scripts/check/` + `scripts/quality/`) wired across **9 gate-running jobs** in `.github/workflows/ci.yml` (`lint`, `quality-gate`, `quality-extended`, `docs-sync-strict`, `i18n-ui-coverage`, `i18n`, `pr-test-policy`, `test-vitest`, `sonarqube`), plus the `quality.yml` fast-gates job (PR→`release/**`) and diff --git a/README.md b/README.md index 9aa175ff43..9b01f08e2b 100644 --- a/README.md +++ b/README.md @@ -7,19 +7,19 @@ # 🚀 OmniRoute — The Free AI Gateway -OmniRoute — Never stop coding. Every AI tool → 291 providers — 90+ free — through one endpoint. Claude Code, Codex, Cursor, Cline, Copilot & Antigravity into FREE Claude / GPT / Gemini with auto-fallback. RTK + Caveman stacked compression saves 15–95% tokens (~89% avg) — never hit limits. 291 AI providers · 90+ free tiers · ~1.53B free tokens/mo · 19 routing strategies · $0 to start. +OmniRoute — Never stop coding. Every AI tool → 338 providers — 90+ free — through one endpoint. Claude Code, Codex, Cursor, Cline, Copilot & Antigravity into FREE Claude / GPT / Gemini with auto-fallback. RTK + Caveman stacked compression saves 15–95% tokens (~89% avg) — never hit limits. 338 AI providers · 90+ free tiers · ~1.51B free tokens/mo · 19 routing strategies · $0 to start.
-## 💰 ~1.53B Free Tokens / Month +## 💰 ~1.51B Free Tokens / Month
-> Stacking free tiers by hand is painful — dozens of SDKs, dozens of rate limits, and no idea how much you actually have. OmniRoute aggregates the **documented** free tiers of **43 provider pools / 516 models** into one honest number and shows it live on the dashboard (`/dashboard/free-tiers`). +> Stacking free tiers by hand is painful — dozens of SDKs, dozens of rate limits, and no idea how much you actually have. OmniRoute aggregates the **documented** free tiers of **42 provider pools / 495 models** into one honest number and shows it live on the dashboard (`/dashboard/free-tiers`). -OmniRoute free-tier budget card: ~1.53B free tokens per month steady, up to ~2.15B in the first month with signup credits, from the documented free tiers of 43 provider pools / 516 models behind one endpoint. Honest pool-deduped math — each shared pool counted once (counting every rate limit 24/7 would read ~10B; not published), 15 providers ToS-flagged so you decide. Budget bar of the countable free pools with per-model grid (Mistral Large 3 1B, GPT-4o mini 150M, Gemini 2.5 Flash 60M … Claude Sonnet 4.5 25K), one-time first-month signup credits (vertex 300M, agentrouter 200M, predibase 25M, together 25M, glm-cn 20M, doubao 15M, ai21 10M, longcat 10M, deepseek 5M, hyperbolic 5M, nscale 5M), plus permanently-free no-token-cap providers (SiliconFlow, Z.AI GLM-Flash, Kilo, OpenCode Zen, baidu …) and a $10 OpenRouter top-up unlocking +24M/mo — surfaced separately so they never inflate the headline. Live used/remaining on /dashboard/free-tiers. +OmniRoute free-tier budget card: ~1.51B free tokens per month steady, up to ~2.13B in the first month with signup credits, from the documented free tiers of 42 provider pools / 495 models behind one endpoint. Honest pool-deduped math — each shared pool counted once (counting every rate limit 24/7 would read ~10B; not published), 15 providers ToS-flagged so you decide. Budget bar of the countable free pools with per-model grid (Mistral Large 3 1B, GPT-4o mini 150M, Gemini 2.5 Flash 60M … Claude Sonnet 4.5 25K), one-time first-month signup credits (vertex 300M, agentrouter 200M, predibase 25M, together 25M, glm-cn 20M, doubao 15M, ai21 10M, longcat 10M, deepseek 5M, hyperbolic 5M, nscale 5M), plus permanently-free no-token-cap providers (SiliconFlow, Z.AI GLM-Flash, Kilo, OpenCode Zen, baidu …) and a $10 OpenRouter top-up unlocking +24M/mo — surfaced separately so they never inflate the headline. Live used/remaining on /dashboard/free-tiers. > Animated summary of the live `/dashboard/free-tiers` page. Full methodology (pool dedupe, credit tiers, provider terms): **[docs/reference/FREE_TIERS.md](docs/reference/FREE_TIERS.md)**. > @@ -81,7 +81,7 @@ ⚙️ Features 🎯 Combos - 🌐 Providers + 🌐 Providers 🔌 CLI & MCP @@ -190,7 +190,7 @@ curl http://localhost:20128/v1/chat/completions \ -The Promise — One endpoint. 291 providers. Never stop building — OmniRoute picks the cheapest one that works. Six pillars: Never hit limits (auto-fallback across 291 providers in milliseconds, zero downtime) · Save up to 95% tokens (RTK + Caveman stacked compression cuts 15–95%, ~89% avg on tool-heavy sessions) · $0 to start (90+ free tiers, 40+ free forever — no card needed) · Every tool works (33 coding agents through one config) · One endpoint (OpenAI ↔ Claude ↔ Gemini ↔ Responses API at /v1) · Production-grade (circuit breakers, TLS stealth, MCP 105 tools, A2A, memory, guardrails, evals — 25,000+ tests). +The Promise — One endpoint. 338 providers. Never stop building — OmniRoute picks the cheapest one that works. Six pillars: Never hit limits (auto-fallback across 338 providers in milliseconds, zero downtime) · Save up to 95% tokens (RTK + Caveman stacked compression cuts 15–95%, ~89% avg on tool-heavy sessions) · $0 to start (90+ free tiers, 40+ free forever — no card needed) · Every tool works (33 coding agents through one config) · One endpoint (OpenAI ↔ Claude ↔ Gemini ↔ Responses API at /v1) · Production-grade (circuit breakers, TLS stealth, MCP 105 tools, A2A, memory, guardrails, evals — 25,000+ tests).

@@ -409,7 +409,7 @@ All **19** strategies — mix & match per combo step: 17 auto - 12-factor live scoring across every connection 🤖 + 14-factor live scoring across every connection 🤖 18 @@ -423,7 +423,7 @@ All **19** strategies — mix & match per combo step: -The Auto-Combo engine scores every candidate on **12 factors** (health, quota, cost, latency, success rate, freshness…) — see [`docs/routing/AUTO-COMBO.md`](docs/routing/AUTO-COMBO.md). +The Auto-Combo engine scores every candidate on **14 factors** (health, quota, cost, latency, success rate, freshness…) — see [`docs/routing/AUTO-COMBO.md`](docs/routing/AUTO-COMBO.md). ## @@ -441,7 +441,7 @@ All **19** strategies — mix & match per combo step: -What sets OmniRoute apart — comparison table vs 9router, OpenRouter, CLIProxyAPI and LiteLLM across 13 capabilities. OmniRoute: 291 providers, 90+ free providers built-in, 19 routing strategies, 12-engine token compression, built-in MCP server with 105 tools, A2A agent protocol, persistent memory, guardrails, cloud agents, TLS fingerprint stealth, Desktop/Termux/PWA, 43 i18n UI locales, 100% MIT self-hosted. OmniRoute is the only one with the full set; competitors show a mix of checks, partials and crosses. Verified from each project's docs. +What sets OmniRoute apart — comparison table vs 9router, OpenRouter, CLIProxyAPI and LiteLLM across 13 capabilities. OmniRoute: 338 providers, 90+ free providers built-in, 19 routing strategies, 12-engine token compression, built-in MCP server with 105 tools, A2A agent protocol, persistent memory, guardrails, cloud agents, TLS fingerprint stealth, Desktop/Termux/PWA, 43 i18n UI locales, 100% MIT self-hosted. OmniRoute is the only one with the full set; competitors show a mix of checks, partials and crosses. Verified from each project's docs. 📊 Full methodology & per-feature detail vs 9router, OpenRouter, CLIProxyAPI & LiteLLM → [`docs/comparison/OMNIROUTE_VS_ALTERNATIVES.md`](docs/comparison/OMNIROUTE_VS_ALTERNATIVES.md) @@ -515,7 +515,7 @@ Pix copia-e-cola: - **🖼️ New endpoints** — `/v1/ocr` (Mistral OCR) and `/v1/audio/translations` (Whisper-style) round out the media surface. → [API Reference](docs/reference/API_REFERENCE.md) - **🎨 Image / video / audio generation** — one API for media: xAI Grok Imagine & Novita AI video, ComfyUI, Freepik, Adobe Firefly, Microsoft Designer, Google Imagen, Segmind, EdgeTTS. → [API Reference](docs/reference/API_REFERENCE.md) - **🌍 Deployment & ops** — reverse-proxy `basePath`, browser-language auto-detect, per-key device tracking, root-less MITM trust, zh-TW localization. → [Environment](docs/reference/ENVIRONMENT.md) -- **🤝 More providers & agents** — Cursor Cloud Agent, Grok Build (xAI) with browser + OAuth login, Ollama first-class card, Claude Opus 5 & Sonnet 5, Kimi official partnership (Code/Web/Moonshot), Zed, Requesty, SenseNova, Yuanbao, Agnes AI… and a refreshed **291-provider catalog**. → [Providers](docs/reference/PROVIDER_REFERENCE.md) +- **🤝 More providers & agents** — Cursor Cloud Agent, Grok Build (xAI) with browser + OAuth login, Ollama first-class card, Claude Opus 5 & Sonnet 5, Kimi official partnership (Code/Web/Moonshot), Zed, Requesty, SenseNova, Yuanbao, Agnes AI… and a refreshed **338-provider catalog**. → [Providers](docs/reference/PROVIDER_REFERENCE.md) - **📡 Routing transparency** — every response carries an `X-OmniRoute-Decision` header naming the strategy/provider/latency that served it, a new `cache-optimized` combo strategy + Auto-Combo `cacheAffinity` factor route repeat requests back to the connection holding the cached prefix, and a read-only `/v1/auto-combo/{channel}/candidates` endpoint exposes an `auto/*` channel's live candidate pool. → [Auto-Combo](docs/routing/AUTO-COMBO.md) - **⚡ Local performance & infra** — one-click local Redis, Cloudflare Workers / Deno Deploy relay deployers, Bifrost & Mux as supervised embedded services. → [Embedded Services](docs/frameworks/EMBEDDED-SERVICES.md) @@ -576,11 +576,11 @@ Pix copia-e-cola:
-## 🌐 291 AI Providers — 90+ Free +## 🌐 338 AI Providers — 90+ Free
-> The most complete catalog of any open-source router: **291 providers**, **90+ with a free tier**, **40+ free forever**. +> The most complete catalog of any open-source router: **338 providers**, **90+ with a free tier**, **40+ free forever**.
@@ -1038,7 +1038,7 @@ same process on one port, so there is no separate CLI-only package today. RuntimeNode.js 22.x / 24.x LTS — >=22.22.2 <23 || >=24.0.0 <27 LanguageTypeScript 6.0 — 100% TypeScript across src/ and open-sse/ (zero any in core since v2.0) FrameworkNext.js 16 + React 19 + Tailwind CSS 4 - Databasebetter-sqlite3 (SQLite, WAL journaling) + LowDB (JSON legacy) — 95 domain modules, 110 migrations + Databasebetter-sqlite3 (SQLite, WAL journaling) + LowDB (JSON legacy) — 95 domain modules, 144 migrations MemorySQLite FTS5 full-text + int8-quantized vector embeddings, typed decay SchemasZod 4 — MCP tool I/O validation + API contracts ProtocolsMCP (stdio / HTTP / SSE) + A2A v0.3 (JSON-RPC 2.0 + SSE) @@ -1099,9 +1099,9 @@ same process on one port, so there is no separate CLI-only package today. Compression Rules FormatJSON rule-pack schemas for Caveman and RTK filters Compression Language PacksLanguage detection and Caveman rule-pack authoring Resilience GuideCircuit breakers, cooldowns, queue, anti-thundering herd, TLS spoofing - Auto-Combo Engine12-factor scoring, mode packs, self-healing + Auto-Combo Engine14-factor scoring, mode packs, self-healing Proxy Guide3-level proxy system, 1proxy marketplace, registry CRUD - Free Tiers25+ free API providers consolidated directory + Free Tiers90+ free providers consolidated directory (42 documented token pools / 495 models) Features GalleryVisual dashboard tour with screenshots Codebase DocumentationBeginner-friendly codebase walkthrough @@ -1112,7 +1112,7 @@ same process on one port, so there is no separate CLI-only package today. DocumentDescription API ReferenceAll endpoints with examples OpenAPI SpecOpenAPI 3.0 specification - MCP Server104 MCP tools, IDE configs, Python/TS/Go clients + MCP Server105 MCP tools, IDE configs, Python/TS/Go clients MCP Server GuideMCP installation, transports, and tool reference A2A ServerJSON-RPC 2.0 protocol, skills, streaming, task mgmt A2A Server GuideA2A agent card, tasks, skills, and streaming @@ -1126,7 +1126,7 @@ same process on one port, so there is no separate CLI-only package today. Branching & Release ModelWhere PRs target (release/*), what main and tags mean ChangelogFull per-version release history Security PolicyVulnerability reporting and security practices - i18n Guide40+ language support, translation workflow, RTL + i18n Guide43-language support, translation workflow, RTL Release ChecklistPre-release validation steps Coverage PlanTest coverage strategy and 25,000+ test suite @@ -1269,7 +1269,7 @@ A heartfelt thank-you to the people who fund OmniRoute out of their own pocket
-## 👥 500+ Contributors +## 👥 320+ Contributors
diff --git a/docs/README.md b/docs/README.md index 96834709ab..3244b9be00 100644 --- a/docs/README.md +++ b/docs/README.md @@ -111,7 +111,7 @@ Pluggable subsystems exposed to clients, agents, and operators. Combo routing, scoring, and replay. -- [AUTO-COMBO.md](routing/AUTO-COMBO.md) — Auto-Combo (multi-factor scoring, 17 strategies). +- [AUTO-COMBO.md](routing/AUTO-COMBO.md) — Auto-Combo (multi-factor scoring, 19 strategies). - [QUOTA_SHARE.md](routing/QUOTA_SHARE.md) — quota sharing engine. - [REASONING_REPLAY.md](routing/REASONING_REPLAY.md) — reasoning replay cache. diff --git a/docs/architecture/ARCHITECTURE.md b/docs/architecture/ARCHITECTURE.md index e99e9b8262..b41bc81297 100644 --- a/docs/architecture/ARCHITECTURE.md +++ b/docs/architecture/ARCHITECTURE.md @@ -17,7 +17,7 @@ It provides a single OpenAI-compatible endpoint (`/v1/*`) and routes traffic acr Core capabilities: -- OpenAI-compatible API surface for CLI/tools (271 providers, 89 executors) +- OpenAI-compatible API surface for CLI/tools (338 providers, 101 executors) - Request/response translation across provider formats - Model combo fallback (multi-model sequence) - Structured combo steps (`provider + model + connection`) with runtime ordering by `compositeTiers` @@ -365,7 +365,7 @@ relying on a static combo definition. It powers the `auto/*` model prefix family Key capabilities: -- **17 routing strategies** (priority, weighted, fill-first, round-robin, P2C, random, +- **19 routing strategies** (priority, weighted, fill-first, round-robin, P2C, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, **auto**, lkgp, context-optimized, context-relay, **fusion**, plus a fallback path) — auto is the headline addition in v3.8.0; `fusion` (panel fan-out + judge synthesis, diff --git a/docs/architecture/CODEBASE_DOCUMENTATION.md b/docs/architecture/CODEBASE_DOCUMENTATION.md index 07544bef2c..e12317863b 100644 --- a/docs/architecture/CODEBASE_DOCUMENTATION.md +++ b/docs/architecture/CODEBASE_DOCUMENTATION.md @@ -452,12 +452,12 @@ open-sse/ ├── types.d.ts ├── config/ Provider registries, header profiles, identity, … ├── handlers/ Request handlers (chat, embeddings, audio, image, …) -├── executors/ 89 provider-specific HTTP executors +├── executors/ 101 provider-specific HTTP executors ├── translator/ Format conversion (OpenAI ↔ Claude ↔ Gemini ↔ Cursor ↔ Kiro) ├── transformer/ Responses API ↔ Chat Completions stream transformer ├── services/ 80+ service modules (combos, fallback, quotas, identity, …) ├── utils/ Streaming helpers, TLS client, AWS SigV4, proxy fetch, … -└── mcp-server/ MCP server (3 transports, 32 scopes, 99 tools) +└── mcp-server/ MCP server (3 transports, 31 scopes, 105 tools) ``` ### 4.1 `open-sse/handlers/` @@ -482,7 +482,7 @@ open-sse/ ### 4.2 `open-sse/executors/` -84 provider executors, each extending `BaseExecutor` (`base.ts`): +101 provider executors, each extending `BaseExecutor` (`base.ts`): `antigravity`, `azure-openai`, `blackbox-web`, `chatgpt-web`, `cliproxyapi`, `cloudflare-ai`, `codex`, `commandCode`, `cursor`, `default`, `devin-cli`, @@ -491,7 +491,7 @@ open-sse/ (shared identity helper) and `index.ts` (registry). > Note: providers not listed here are served by `default.ts` using the generic -> OpenAI-compatible executor. The full provider catalog (268 entries) lives in +> OpenAI-compatible executor. The full provider catalog (338 providers) lives in > `src/shared/constants/providers.ts`. ### 4.3 `open-sse/translator/` @@ -524,7 +524,7 @@ Highlights (full list under `open-sse/services/`): | Concern | Files | | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Combo routing | `combo.ts` (17 strategies), `comboConfig.ts`, `comboMetrics.ts`, `comboManifestMetrics.ts`, `comboAgentMiddleware.ts` | +| Combo routing | `combo.ts` (19 strategies), `comboConfig.ts`, `comboMetrics.ts`, `comboManifestMetrics.ts`, `comboAgentMiddleware.ts` | | Auto Combo engine | `autoCombo/` — `engine.ts`, `scoring.ts`, `taskFitness.ts`, `virtualFactory.ts`, `modePacks.ts`, `autoPrefix.ts`, `persistence.ts`, `providerDiversity.ts`, `providerRegistryAccessor.ts`, `routerStrategy.ts`, `selfHealing.ts`, `index.ts` | | Resilience | `accountFallback.ts` (cooldown + lockout), `errorClassifier.ts`, `emergencyFallback.ts`, `rateLimitManager.ts`, `rateLimitSemaphore.ts`, `accountSemaphore.ts`, `accountSelector.ts` | | Quotas | `quotaMonitor.ts`, `quotaPreflight.ts`, `bailianQuotaFetcher.ts`, `codexQuotaFetcher.ts`, `deepseekQuotaFetcher.ts`, `openrouterQuotaFetcher.ts`, `openrouterFreeWindow.ts`, `crofUsageFetcher.ts`, `antigravityCredits.ts` | @@ -544,7 +544,7 @@ Highlights (full list under `open-sse/services/`): 5 compression tools, 3 memory tools, 4 skills tools, plus advanced tools added through `advancedTools.ts`). - **3 transports**: stdio, HTTP Streamable, SSE. -- **13 scopes** declared in `src/shared/constants/mcpScopes.ts`. +- **31 scopes** declared in `src/shared/constants/mcpScopes.ts`. - Audit table: `mcp_tool_audit` (populated by `audit.ts`). - Files: `server.ts`, `index.ts`, `httpTransport.ts`, `audit.ts`, `scopeEnforcement.ts`, `runtimeHeartbeat.ts`, `descriptionCompressor.ts`, `schemas/{tools, a2a, audit, index}.ts`, diff --git a/docs/architecture/MONITORING_SECTIONS.md b/docs/architecture/MONITORING_SECTIONS.md index ca03a1136c..3fa9657caa 100644 --- a/docs/architecture/MONITORING_SECTIONS.md +++ b/docs/architecture/MONITORING_SECTIONS.md @@ -142,5 +142,5 @@ Namespaces added by Group B: | `sidebar.costsOverview` | Costs overview item | | `activity.*` | All Activity page strings (title, verbs, filters, empty state) | -Source-of-truth locales: `pt-BR` and `en`. All other 39 locales fall back to +Source-of-truth locales: `pt-BR` and `en`. All other 41 locales fall back to English via the `next-intl` fallback mechanism (configured in `src/i18n/config.ts`). diff --git a/docs/architecture/QUALITY_GATES.md b/docs/architecture/QUALITY_GATES.md index 34c05d0699..1501bb60f7 100644 --- a/docs/architecture/QUALITY_GATES.md +++ b/docs/architecture/QUALITY_GATES.md @@ -205,7 +205,7 @@ Runs after `build`. Blocks merge on failure. | Suite | Validates | Blocking | | ---------------- | ------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | -| `test:vitest` | MCP server (94 tools), autoCombo, cache — vitest runner | Yes | +| `test:vitest` | MCP server (105 tools), autoCombo, cache — vitest runner | Yes | | `test:vitest:ui` | UI component tests — vitest runner | **Blocking** — pre-existing failures are explicitly excluded in `vitest.config.ts`; new failures fail the job | ### Nightly workflows (scheduled, advisory) diff --git a/docs/architecture/REPOSITORY_MAP.md b/docs/architecture/REPOSITORY_MAP.md index 20238334d1..5cee8f04a0 100644 --- a/docs/architecture/REPOSITORY_MAP.md +++ b/docs/architecture/REPOSITORY_MAP.md @@ -182,7 +182,7 @@ src/ | `compliance/` | Audit log + provider audit — see `docs/security/COMPLIANCE.md` | | `compression/` | Compression engine glue (engines live in `open-sse/services/compression/`) | | `config/` | Runtime config helpers | -| `db/` | 95+ domain DB modules + 110+ migrations (always go through here for SQLite) | +| `db/` | 95+ domain DB modules + 144 migrations (always go through here for SQLite) | | `quota/` | Quota Sharing Engine: `dimensions.ts` (types/Zod), `types.ts` (QuotaStore interface), `sqliteQuotaStore.ts`, `redisQuotaStore.ts`, `storeFactory.ts`, `fairShare.ts`, `burnRate.ts`, `planResolver.ts`, `planRegistry.ts`, `saturationSignals.ts`, `enforce.ts`, `spendRecorder.ts` — see `docs/routing/QUOTA_SHARE.md` | | `radar/` | Radar free-model catalog client: `feedSchema.ts`, `pinnedKeys.ts`, `verify.ts`, `sync.ts`, `applyFeed.ts`, `index.ts` (`getRadarCatalog()`) — see `docs/frameworks/RADAR.md` | | `display/` | UI formatting helpers (cost, latency, etc.) | @@ -209,7 +209,7 @@ src/ | `cacheLayer.ts`, `idempotencyLayer.ts` | Request caching + idempotency | | (~30 more top-level files) | Specialized helpers (logEnv, modelsDevSync, piiSanitizer, etc.) | -### `src/db/` — Database (94 modules + 106 migrations) +### `src/db/` — Database (117 modules + 144 migrations) | Subdir | Purpose | | ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | @@ -244,9 +244,9 @@ src/ | Module | Purpose | | -------------------------------- | ---------------------------------------------------------------------- | -| `constants/providers.ts` | **236 providers** with Zod validation (source of truth) | +| `constants/providers.ts` | **338 providers** with Zod validation (source of truth) | | `constants/cliTools.ts` | External CLI tool registry | -| `constants/routingStrategies.ts` | **17 routing strategies** with priorities | +| `constants/routingStrategies.ts` | **19 routing strategies** with priorities | | `constants/publicApiRoutes.ts` | Routes that require Bearer (vs management) auth | | `constants/upstreamHeaders.ts` | Header denylist for upstream requests | | `validation/schemas.ts` | ~80 Zod schemas (single source of truth for API contracts) | @@ -270,7 +270,7 @@ open-sse/ ├── translator/ # Format converters (9 request, 9 response, 9 helpers) ├── transformer/ # Responses API ↔ Chat Completions (TransformStream) ├── services/ # ~80+ service modules (combo, accountFallback, autoCombo, reasoningCache, claude code/chatgpt stealth, modelDeprecation, taskAwareRouter, workflowFSM, etc.) -├── mcp-server/ # MCP server (99 tools, 3 transports, 32 scopes) +├── mcp-server/ # MCP server (105 tools, 3 transports, 31 scopes) ├── config/ # Provider/model registries, header config, model aliases ├── utils/ # TLS client, proxy fetch/dispatcher, network helpers ├── index.ts # Workspace entry @@ -398,16 +398,16 @@ open-sse/ | `COVERAGE_PLAN.md` | Coverage goals and current state | | `FREE_TIERS.md` | Curated free-tier providers (48+ free + 11 OAuth) | | `CLI-TOOLS.md` | External CLI integrations + Internal OmniRoute CLI | -| `I18N.md` | i18n architecture, adding a language, 30 locales | +| `I18N.md` | i18n architecture, adding a language, 43 locales | | `UNINSTALL.md` | Clean uninstall steps | -| `PROVIDER_REFERENCE.md` | **Auto-generated** catalog of 236 providers (regen: `npm run gen:provider-reference`) | +| `PROVIDER_REFERENCE.md` | **Auto-generated** catalog of 338 providers (regen: `npm run gen:provider-reference`) | ### Subsystem deep-dives | Doc | Purpose | | -------------------------- | ------------------------------------------------------------------- | -| `MCP-SERVER.md` | MCP server: 99 tools, 3 transports, 32 scopes, REST endpoints | -| `A2A-SERVER.md` | A2A v0.3: JSON-RPC, 5 skills, REST helpers, agent card | +| `MCP-SERVER.md` | MCP server: 105 tools, 3 transports, 31 scopes, REST endpoints | +| `A2A-SERVER.md` | A2A v0.3: JSON-RPC, 6 skills, REST helpers, agent card | | `AGENT_PROTOCOLS_GUIDE.md` | Unified guide: A2A vs ACP vs Cloud Agents | | `CLOUD_AGENT.md` | Codex Cloud / Devin / Jules orchestration | | `SKILLS.md` | Skills framework (built-in + marketplace + SkillsSH + sandbox) | diff --git a/docs/frameworks/ACP.md b/docs/frameworks/ACP.md index 3a7b386220..997fb828b0 100644 --- a/docs/frameworks/ACP.md +++ b/docs/frameworks/ACP.md @@ -543,7 +543,7 @@ const agents = detectInstalledAgents(); ## What's Next? - **[API Reference](../reference/API_REFERENCE.md)** — REST API endpoints -- **[Provider Reference](../reference/PROVIDER_REFERENCE.md)** — All 226 providers +- **[Provider Reference](../reference/PROVIDER_REFERENCE.md)** — All 338 providers - **[MCP Server](./MCP-SERVER.md)** — Model Context Protocol integration - **[A2A Server](./A2A-SERVER.md)** — Agent-to-Agent protocol - **[Cloud Agent](./CLOUD_AGENT.md)** — Cloud-based agents diff --git a/docs/frameworks/AGENTBRIDGE.md b/docs/frameworks/AGENTBRIDGE.md index 76949f46fe..85cba9a28c 100644 --- a/docs/frameworks/AGENTBRIDGE.md +++ b/docs/frameworks/AGENTBRIDGE.md @@ -22,7 +22,7 @@ When an IDE agent (e.g., GitHub Copilot, Cursor, Claude Code) makes an API call, This means you can: -- **Reroute any agent to any provider**: Copilot talking to OpenAI? Redirect it to Anthropic Claude, Gemini, or any of OmniRoute's 226+ providers. +- **Reroute any agent to any provider**: Copilot talking to OpenAI? Redirect it to Anthropic Claude, Gemini, or any of OmniRoute's 338 providers. - **Apply model mappings**: `gemini-3-flash` → `claude-sonnet-4.7` transparently at the handler level. - **Observe all agent traffic**: every intercepted request is published to the [Traffic Inspector](./TRAFFIC_INSPECTOR.md). - **Apply OmniRoute resilience**: combo routing, circuit breakers, fallbacks, and cost tracking work for IDE agent traffic too. diff --git a/docs/frameworks/AGENT_PROTOCOLS_GUIDE.md b/docs/frameworks/AGENT_PROTOCOLS_GUIDE.md index 305c398ced..46354225a5 100644 --- a/docs/frameworks/AGENT_PROTOCOLS_GUIDE.md +++ b/docs/frameworks/AGENT_PROTOCOLS_GUIDE.md @@ -183,7 +183,7 @@ Both have "long-running tasks" but at different layers: curl http://localhost:20128/.well-known/agent.json ``` -Returns the Agent Card with all 5 skills, transports, and version. +Returns the Agent Card with all 6 skills, transports, and version. ### Call OmniRoute as an A2A agent diff --git a/docs/frameworks/MCP-SERVER.md b/docs/frameworks/MCP-SERVER.md index 7933941011..b9ea9e6b36 100644 --- a/docs/frameworks/MCP-SERVER.md +++ b/docs/frameworks/MCP-SERVER.md @@ -6,7 +6,7 @@ lastUpdated: 2026-06-28 # OmniRoute MCP Server Documentation -> Model Context Protocol server with 104 tools across routing, cache, compression, memory, skills, proxy, pool, and context source operations. +> Model Context Protocol server with 105 tools across routing, cache, compression, memory, skills, proxy, pool, and context source operations. > > Source of truth: `open-sse/mcp-server/server.ts` computes **104 unique tools** with `countUniqueMcpTools()`: 42 canonical definitions (including the six CCR lifecycle tools and the agent-skills trio), plus memory (3), skills (4), GitHub skills (3), pool (6), gamification (8), plugins (8), Notion (6), Obsidian (22), and two RTK-only compression tools. @@ -369,7 +369,7 @@ MCP tool, prompt, and resource registries can compress descriptions at registrat Description compression shrinks each tool's metadata; **tool-cardinality reduction** goes one step further by reducing _how many_ tools are announced at all. Advertising fewer tools in the `tools/list` manifest cuts the per-request token cost the client's model pays for the tool catalog ("layer 5" compression). The implementation is a pure, stateless filter in `open-sse/mcp-server/toolCardinality.ts` (`reduceToolManifest`), wired into the registration loop in `createMcpServer()` (`open-sse/mcp-server/server.ts`). -**Opt-in, off by default.** The filter only runs when at least one of two environment variables is set; with neither set, all 104 tools are announced unchanged. +**Opt-in, off by default.** The filter only runs when at least one of two environment variables is set; with neither set, all 105 tools are announced unchanged. | Variable | Mode | | :--------------- | :-------------------------------------------------------------------------------------- | diff --git a/docs/frameworks/OPEN_SSE_ARCHITECTURE.md b/docs/frameworks/OPEN_SSE_ARCHITECTURE.md index dc525245dd..83ebf35a95 100644 --- a/docs/frameworks/OPEN_SSE_ARCHITECTURE.md +++ b/docs/frameworks/OPEN_SSE_ARCHITECTURE.md @@ -228,7 +228,7 @@ export async function handleComboChat(body, comboId): Promise { } ``` -Supports **17 routing strategies** (see `src/shared/constants/routingStrategies.ts`): +Supports **19 routing strategies** (see `src/shared/constants/routingStrategies.ts`): | Strategy | Behavior | | ------------------- | ------------------------------------------------------------------------- | @@ -252,7 +252,7 @@ Supports **17 routing strategies** (see `src/shared/constants/routingStrategies. ### base.ts (1170 LOC) -The **abstract executor** that all 67 executors extend. It contains: +The **abstract executor** that all 101 executors extend. It contains: - `buildUrl()` — default URL construction (subclasses override for custom) - `buildHeaders()` — default headers (auth, content-type) @@ -368,7 +368,7 @@ const result = await executor.execute({ }); ```` -The factory is generated from `config/providerRegistry.ts` which lists all 212+ providers and their executor class. +The factory is generated from `config/providerRegistry.ts` which lists all 338 providers and their executor class. --- @@ -408,7 +408,7 @@ Common translations: - **30+ tools** (provider management, combos, memory, cache, compression, 1proxy, skills) - **3 transports**: stdio, SSE, Streamable HTTP -- **13 scopes** for fine-grained authorization +- **31 scopes** for fine-grained authorization ### Tool Registration @@ -487,7 +487,7 @@ This handles: | File | Purpose | | ----------------------------- | --------------------------------- | -| `providerRegistry.ts` | 212+ provider definitions | +| `providerRegistry.ts` | 338 provider definitions | | `providerModels.ts` | Model aliases, format mapping | | `constants.ts` | Timeouts, limits, status codes | | `defaultThinkingSignature.ts` | Default Claude thinking signature | diff --git a/docs/guides/I18N.md b/docs/guides/I18N.md index 1cc5a7f088..87b06395de 100644 --- a/docs/guides/I18N.md +++ b/docs/guides/I18N.md @@ -88,9 +88,9 @@ README variants) are not yet handled by the new pipeline and are still used. ### Source of Truth - **UI strings**: `src/i18n/messages/en.json` (English source, ~2800 keys) -- **Locale files**: `src/i18n/messages/{locale}.json` (30 translations) +- **Locale files**: `src/i18n/messages/{locale}.json` (43 translations) - **Framework**: `next-intl` with cookie-based locale resolution -- **Config**: `src/i18n/config.ts` — defines all 30 locales, language names, flags +- **Config**: `src/i18n/config.ts` — defines all 43 locales, language names, flags ### Runtime Flow @@ -304,7 +304,7 @@ invocation. Use `config lang set` to persist. 43 locale files ship in `bin/cli/locales/`. Full translations: `en`, `pt-BR`. Scaffold-only (all keys fall back to `en`): `bn`, `gu`, `he`, `in`, `mr`, `ms`, `phi`, `sw`, `ta`, `te`, `ur`. -All other 29 locales have `common` + `program` keys translated. +All other 30 locales have `common` + `program` keys translated. ### Adding a new CLI locale @@ -533,7 +533,7 @@ python3 scripts/i18n/validate_translation.py quick -l '${{ matrix.lang }}' ``` src/i18n/ -├── config.ts # Locale definitions (30 locales, RTL config) +├── config.ts # Locale definitions (43 locales, RTL config) ├── request.ts # Runtime locale resolution └── messages/ ├── en.json # Source of truth (~2800 keys) diff --git a/docs/i18n/ar/llm.txt b/docs/i18n/ar/llm.txt index d157540caa..454f5dcd4d 100644 --- a/docs/i18n/ar/llm.txt +++ b/docs/i18n/ar/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/az/llm.txt b/docs/i18n/az/llm.txt index 075229caa5..d829905bcd 100644 --- a/docs/i18n/az/llm.txt +++ b/docs/i18n/az/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/bg/llm.txt b/docs/i18n/bg/llm.txt index 075229caa5..d829905bcd 100644 --- a/docs/i18n/bg/llm.txt +++ b/docs/i18n/bg/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/bn/llm.txt b/docs/i18n/bn/llm.txt index 686cf2afbc..e0a399b644 100644 --- a/docs/i18n/bn/llm.txt +++ b/docs/i18n/bn/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/cs/llm.txt b/docs/i18n/cs/llm.txt index 5dbaf03e99..922e176440 100644 --- a/docs/i18n/cs/llm.txt +++ b/docs/i18n/cs/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/da/llm.txt b/docs/i18n/da/llm.txt index 29093c0fce..fc4d0b2466 100644 --- a/docs/i18n/da/llm.txt +++ b/docs/i18n/da/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/de/llm.txt b/docs/i18n/de/llm.txt index 175fbf938b..c013784a40 100644 --- a/docs/i18n/de/llm.txt +++ b/docs/i18n/de/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/es/llm.txt b/docs/i18n/es/llm.txt index 01d19c8093..c383e19104 100644 --- a/docs/i18n/es/llm.txt +++ b/docs/i18n/es/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/fa/llm.txt b/docs/i18n/fa/llm.txt index 4ee1ea1f79..fe7f03f67d 100644 --- a/docs/i18n/fa/llm.txt +++ b/docs/i18n/fa/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/fi/llm.txt b/docs/i18n/fi/llm.txt index 777b728cb9..11273b2f23 100644 --- a/docs/i18n/fi/llm.txt +++ b/docs/i18n/fi/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/fr/llm.txt b/docs/i18n/fr/llm.txt index 83e126582a..e1c32578f8 100644 --- a/docs/i18n/fr/llm.txt +++ b/docs/i18n/fr/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/gu/llm.txt b/docs/i18n/gu/llm.txt index 909f1818f2..26a0640805 100644 --- a/docs/i18n/gu/llm.txt +++ b/docs/i18n/gu/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/he/llm.txt b/docs/i18n/he/llm.txt index a229eeb907..39c47704a3 100644 --- a/docs/i18n/he/llm.txt +++ b/docs/i18n/he/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/hi/llm.txt b/docs/i18n/hi/llm.txt index e92f5650b4..27fddd2e19 100644 --- a/docs/i18n/hi/llm.txt +++ b/docs/i18n/hi/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/hu/llm.txt b/docs/i18n/hu/llm.txt index 40b0dd095c..a1d63275bb 100644 --- a/docs/i18n/hu/llm.txt +++ b/docs/i18n/hu/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/id/llm.txt b/docs/i18n/id/llm.txt index 8b61036b68..377391d905 100644 --- a/docs/i18n/id/llm.txt +++ b/docs/i18n/id/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/in/llm.txt b/docs/i18n/in/llm.txt index 86bf32e43e..bbd35be114 100644 --- a/docs/i18n/in/llm.txt +++ b/docs/i18n/in/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/it/llm.txt b/docs/i18n/it/llm.txt index 0341b59136..d14785348e 100644 --- a/docs/i18n/it/llm.txt +++ b/docs/i18n/it/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/ja/llm.txt b/docs/i18n/ja/llm.txt index 92af0f6bae..e6c52a40c6 100644 --- a/docs/i18n/ja/llm.txt +++ b/docs/i18n/ja/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/ko/llm.txt b/docs/i18n/ko/llm.txt index 74258069db..e0e5de2910 100644 --- a/docs/i18n/ko/llm.txt +++ b/docs/i18n/ko/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/mr/llm.txt b/docs/i18n/mr/llm.txt index 9f79456c69..5a9ce36db4 100644 --- a/docs/i18n/mr/llm.txt +++ b/docs/i18n/mr/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/ms/llm.txt b/docs/i18n/ms/llm.txt index 9c5b6b9760..4c7cee1944 100644 --- a/docs/i18n/ms/llm.txt +++ b/docs/i18n/ms/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/nl/llm.txt b/docs/i18n/nl/llm.txt index 1b2b3435a3..77a8c8a29c 100644 --- a/docs/i18n/nl/llm.txt +++ b/docs/i18n/nl/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/no/llm.txt b/docs/i18n/no/llm.txt index f3f4e73e48..992be4b63f 100644 --- a/docs/i18n/no/llm.txt +++ b/docs/i18n/no/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/phi/llm.txt b/docs/i18n/phi/llm.txt index 015777774b..9a248f4be2 100644 --- a/docs/i18n/phi/llm.txt +++ b/docs/i18n/phi/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/pl/llm.txt b/docs/i18n/pl/llm.txt index b2dc9c9fbf..e47af25132 100644 --- a/docs/i18n/pl/llm.txt +++ b/docs/i18n/pl/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/pt-BR/llm.txt b/docs/i18n/pt-BR/llm.txt index c89a273b7d..2ea921f25e 100644 --- a/docs/i18n/pt-BR/llm.txt +++ b/docs/i18n/pt-BR/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/pt/llm.txt b/docs/i18n/pt/llm.txt index b29bbb630c..185f5badfe 100644 --- a/docs/i18n/pt/llm.txt +++ b/docs/i18n/pt/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/ro/llm.txt b/docs/i18n/ro/llm.txt index 1910de17e7..aa9f7446e3 100644 --- a/docs/i18n/ro/llm.txt +++ b/docs/i18n/ro/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/ru/llm.txt b/docs/i18n/ru/llm.txt index 4ed0d7286e..3f008ce8c7 100644 --- a/docs/i18n/ru/llm.txt +++ b/docs/i18n/ru/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/sk/llm.txt b/docs/i18n/sk/llm.txt index 8547faa1df..dda73b9e77 100644 --- a/docs/i18n/sk/llm.txt +++ b/docs/i18n/sk/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/sv/llm.txt b/docs/i18n/sv/llm.txt index 9c6c24b4db..9fcf0081fe 100644 --- a/docs/i18n/sv/llm.txt +++ b/docs/i18n/sv/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/sw/llm.txt b/docs/i18n/sw/llm.txt index 6549c8f5f3..69650fcc5c 100644 --- a/docs/i18n/sw/llm.txt +++ b/docs/i18n/sw/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/ta/llm.txt b/docs/i18n/ta/llm.txt index d1f7d8967f..f2660ef9a5 100644 --- a/docs/i18n/ta/llm.txt +++ b/docs/i18n/ta/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/te/llm.txt b/docs/i18n/te/llm.txt index bb2fee59c7..8e04cb2e9f 100644 --- a/docs/i18n/te/llm.txt +++ b/docs/i18n/te/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/th/llm.txt b/docs/i18n/th/llm.txt index 430442fdb2..2b6ae43aeb 100644 --- a/docs/i18n/th/llm.txt +++ b/docs/i18n/th/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/tr/llm.txt b/docs/i18n/tr/llm.txt index 91092cff9f..0c1c41eb05 100644 --- a/docs/i18n/tr/llm.txt +++ b/docs/i18n/tr/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/uk-UA/llm.txt b/docs/i18n/uk-UA/llm.txt index 7b70b0553a..5a7213ed4a 100644 --- a/docs/i18n/uk-UA/llm.txt +++ b/docs/i18n/uk-UA/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/ur/llm.txt b/docs/i18n/ur/llm.txt index fa189b93fa..95720495f8 100644 --- a/docs/i18n/ur/llm.txt +++ b/docs/i18n/ur/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/vi/llm.txt b/docs/i18n/vi/llm.txt index 21fdc5fc78..b4c9ccbe4d 100644 --- a/docs/i18n/vi/llm.txt +++ b/docs/i18n/vi/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/zh-CN/llm.txt b/docs/i18n/zh-CN/llm.txt index cdd5e17b39..22c88dc845 100644 --- a/docs/i18n/zh-CN/llm.txt +++ b/docs/i18n/zh-CN/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/i18n/zh-TW/llm.txt b/docs/i18n/zh-TW/llm.txt index e5a59d9617..13149ea8d1 100644 --- a/docs/i18n/zh-TW/llm.txt +++ b/docs/i18n/zh-TW/llm.txt @@ -4,7 +4,8 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. + +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,13 +13,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -26,7 +27,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -45,7 +46,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -98,7 +99,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -169,7 +170,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +187,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,11 +213,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -228,7 +229,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -267,8 +268,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -279,15 +280,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -316,7 +317,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -346,13 +347,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -373,7 +374,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -385,7 +386,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -439,7 +440,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -447,7 +448,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -480,10 +481,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/docs/routing/AUTO-COMBO.md b/docs/routing/AUTO-COMBO.md index 1b42354be7..7a0b387191 100644 --- a/docs/routing/AUTO-COMBO.md +++ b/docs/routing/AUTO-COMBO.md @@ -161,11 +161,11 @@ The detection helper lives in `src/lib/combos/modelNameCollision.ts`. ## How It Works (Persisted Auto-Combos) -The Auto-Combo Engine dynamically selects the best provider/model for each request using a **13-factor scoring function** (defined in `open-sse/services/autoCombo/scoring.ts` → `DEFAULT_WEIGHTS`). All weights sum to **1.0**. +The Auto-Combo Engine dynamically selects the best provider/model for each request using a **14-factor scoring function** (defined in `open-sse/services/autoCombo/scoring.ts` → `DEFAULT_WEIGHTS`). Weights form a normalized distribution (custom weights are renormalized by `normalizeScoringWeights()`). -![Auto-Combo 12-factor scoring](../diagrams/exported/auto-combo-12factor.svg) +![Auto-Combo 14-factor scoring](../diagrams/exported/auto-combo-12factor.svg) -> Source: [diagrams/auto-combo-12factor.mmd](../diagrams/auto-combo-12factor.mmd) (regenerate via `npm run docs:render-diagrams`). Diagram/filename predate the `cacheAffinity` factor added by #8008 and still show 12 factors. +> Source: [diagrams/auto-combo-12factor.mmd](../diagrams/auto-combo-12factor.mmd) (regenerate via `npm run docs:render-diagrams`). The filename predates the current factor set; the diagram shows 13 of the 14 factors (missing `sessionAvailability`). | Factor | Default Weight | Description | | :-------------------- | :------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | @@ -179,11 +179,12 @@ The Auto-Combo Engine dynamically selects the best provider/model for each reque | `tierAffinity` | 0.05 | Affinity between the candidate's tier and the manifest-recommended tier | | `specificityMatch` | 0.05 | Match between request specificity (manifest hint) and model tier | | `contextAffinity` | 0.05 | Affinity between the request's context-window need and the model's context window | +| `sessionAvailability` | 0.05 | OAuth session availability of the candidate connection for this session (`getOAuthSessionAvailability()`; non-OAuth connections score 1.0) | | `connectionDensity` | 0.05 | Spreads load across connections of the same provider (anti-concentration) | | `cacheAffinity` | 0.00 | Rendezvous-hash affinity toward the connection likeliest to already hold this request's prompt-cache prefix (`open-sse/services/combo/promptCacheAffinity.ts`); disabled by default (#8008) | | `resetWindowAffinity` | 0.00 | Bias toward connections whose quota reset window is favorable (disabled by default) | -**Sum:** `0.20 + 0.15 + 0.15 + 0.12 + 0.08 + 0.05 + 0.05 + 0.05 + 0.05 + 0.05 + 0.05 + 0.00 + 0.00 = 1.0` (validated by `validateWeights()`). +**Sum:** `0.20 + 0.15 + 0.15 + 0.12 + 0.08 + 0.05 + 0.05 + 0.05 + 0.05 + 0.05 + 0.05 + 0.05 + 0.00 + 0.00 = 1.05` as literally declared in `DEFAULT_WEIGHTS`; user-configured weights are renormalized into a distribution by `normalizeScoringWeights()` before scoring. ## Mode Packs @@ -654,7 +655,7 @@ Including the bare `auto` (default) plus the 6 `AutoVariant` values declared in ## How tiers fit Auto-Combo -The 12-factor scoring function (`open-sse/services/autoCombo/scoring.ts`) treats tier +The 14-factor scoring function (`open-sse/services/autoCombo/scoring.ts`) treats tier membership as two signals: `tierPriority` (0.05) and `tierAffinity` (0.05). See the canonical [scoring factor table](#how-it-works-persisted-auto-combos) above for the full `DEFAULT_WEIGHTS` set — the per-pack overrides (ship-fast/cost-saver/quality-first/ diff --git a/llm.txt b/llm.txt index daed1b645f..470ad5f415 100644 --- a/llm.txt +++ b/llm.txt @@ -1,6 +1,6 @@ # OmniRoute -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (99 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 338 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (105 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -8,13 +8,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.47 +**Current version:** 3.8.50 ## Tech Stack - **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) - **Framework:** Next.js 16 (App Router) with TypeScript 6 -- **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) +- **Database:** SQLite via better-sqlite3 (local, zero-config, 144 migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons - **Auth:** OAuth 2.0 (PKCE) for providers, bcrypt for local user auth @@ -22,7 +22,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Background jobs:** Custom token health check scheduler, 24h model auto-sync - **Streaming:** Server-Sent Events (SSE) for real-time proxy responses - **Proxy engine:** Custom pipeline with format translation, circuit breaker, rate limiting, auto-combo engine -- **i18n:** next-intl with 40+ languages +- **i18n:** next-intl with 43 languages - **Desktop:** Electron (cross-platform: Windows, macOS, Linux) - **Package:** Published on npm (`omniroute`) and Docker Hub (`diegosouzapw/omniroute`) @@ -41,7 +41,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (19 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -94,7 +94,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── configAudit.ts # Configuration auditing │ │ └── responses.ts # Domain response types │ ├── i18n/ # Internationalization -│ │ └── messages/ # 40+ language JSON files +│ │ └── messages/ # 43 language JSON files │ ├── lib/ # Core libraries │ │ ├── a2a/ # Agent-to-Agent v0.3 protocol server │ │ │ ├── skills/ # A2A skills (quotaManagement, smartRouting) @@ -165,7 +165,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (338), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -182,7 +182,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (78 executor modules) +│ ├── executors/ # Provider-specific request executors (101 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -208,11 +208,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (99 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (105 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (32 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (31 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler @@ -224,7 +224,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -263,8 +263,8 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── i18n/ # 43-language translated docs │ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md │ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md -│ ├── frameworks/ # MCP-SERVER.md (99 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md -│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── frameworks/ # MCP-SERVER.md (105 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md │ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md │ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md │ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment @@ -275,15 +275,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.47) +## Key Features (v3.8.50) ### Core Proxy -- **248 AI providers** with automatic format translation +- **338 AI providers** with automatic format translation - **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) - **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -312,7 +312,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -342,13 +342,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 99-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) -- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) +- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (99 Tools) +### MCP Server (105 Tools) -99 tools across modules: **36 base** (health, combos, quotas, routing, cost, models, cache, +105 tools across modules: **43 base** (health, combos, quotas, routing, cost, models, cache, diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**, **obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory: `docs/frameworks/MCP-SERVER.md`. @@ -369,7 +369,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool **Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs ### Internationalization -- 40+ languages for UI (all dashboard pages) +- 43 languages for UI (all dashboard pages) - 40 translated documentation sets in docs/i18n/ - Language switcher in documentation @@ -381,7 +381,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 19 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. @@ -435,7 +435,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (117 domain-specific files, 144 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. 6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. @@ -443,7 +443,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -476,10 +476,10 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool ## v3.8.x Highlights -- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add -- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` -- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown -- **MCP server expanded to 99 tools / 32 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) +- **338-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 105 tools / 31 scopes** (base + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules) - **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks - **Embedded services** manager (install/start/stop bundled services from the dashboard) - **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic diff --git a/package.json b/package.json index 86008e9677..e89911492c 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "omniroute", "version": "3.8.50", - "description": "Unified AI router with 291 providers, RTK+Caveman compression, auto fallback, MCP/A2A, desktop, PWA, and OpenAI-compatible APIs.", + "description": "Unified AI router with 338 providers, RTK+Caveman compression, auto fallback, MCP/A2A, desktop, PWA, and OpenAI-compatible APIs.", "type": "module", "bin": { "omniroute": "bin/omniroute.mjs",