From d11fd9380ba5e0b5c24ec7186bec44366b60ca0c Mon Sep 17 00:00:00 2001 From: Diego Rodrigues de Sa e Souza <8016841+diegosouzapw@users.noreply.github.com> Date: Sat, 11 Jul 2026 04:57:32 -0300 Subject: [PATCH] docs: refresh stale llm.txt facts + relocate design.md to docs/architecture/DESIGN_SYSTEM.md (#6849) * docs: refresh stale llm.txt facts + move design.md to docs/architecture/DESIGN_SYSTEM.md llm.txt was frozen at the v3.8.8 era (177 providers, 37 MCP tools, 14 strategies, 9-factor scoring, 75% coverage gate). Update every factual claim to the current state (248 providers, 94 tools / 30 scopes, 18 strategies, 12-factor scoring, ratchet + 60% floor, TS 6, current docs/ layout) and re-sync the 42 exact-copy i18n mirrors. design.md at the root was a standardization plan whose phases 1-6 all shipped; rewrite its header as a permanent reference and relocate it to docs/architecture/DESIGN_SYSTEM.md per the root-hygiene policy (root = configs + canonical docs only). * docs: add MDX frontmatter to DESIGN_SYSTEM.md (in-app docs pipeline requires it) --- .../root-cleanup-llmtxt-design-system.md | 1 + .../architecture/DESIGN_SYSTEM.md | 15 ++- docs/i18n/ar/llm.txt | 115 +++++++++--------- docs/i18n/az/llm.txt | 115 +++++++++--------- docs/i18n/bg/llm.txt | 115 +++++++++--------- docs/i18n/bn/llm.txt | 115 +++++++++--------- docs/i18n/cs/llm.txt | 115 +++++++++--------- docs/i18n/da/llm.txt | 115 +++++++++--------- docs/i18n/de/llm.txt | 115 +++++++++--------- docs/i18n/es/llm.txt | 115 +++++++++--------- docs/i18n/fa/llm.txt | 115 +++++++++--------- docs/i18n/fi/llm.txt | 115 +++++++++--------- docs/i18n/fr/llm.txt | 115 +++++++++--------- docs/i18n/gu/llm.txt | 115 +++++++++--------- docs/i18n/he/llm.txt | 115 +++++++++--------- docs/i18n/hi/llm.txt | 115 +++++++++--------- docs/i18n/hu/llm.txt | 115 +++++++++--------- docs/i18n/id/llm.txt | 115 +++++++++--------- docs/i18n/in/llm.txt | 115 +++++++++--------- docs/i18n/it/llm.txt | 115 +++++++++--------- docs/i18n/ja/llm.txt | 115 +++++++++--------- docs/i18n/ko/llm.txt | 115 +++++++++--------- docs/i18n/mr/llm.txt | 115 +++++++++--------- docs/i18n/ms/llm.txt | 115 +++++++++--------- docs/i18n/nl/llm.txt | 115 +++++++++--------- docs/i18n/no/llm.txt | 115 +++++++++--------- docs/i18n/phi/llm.txt | 115 +++++++++--------- docs/i18n/pl/llm.txt | 115 +++++++++--------- docs/i18n/pt-BR/llm.txt | 115 +++++++++--------- docs/i18n/pt/llm.txt | 115 +++++++++--------- docs/i18n/ro/llm.txt | 115 +++++++++--------- docs/i18n/ru/llm.txt | 115 +++++++++--------- docs/i18n/sk/llm.txt | 115 +++++++++--------- docs/i18n/sv/llm.txt | 115 +++++++++--------- docs/i18n/sw/llm.txt | 115 +++++++++--------- docs/i18n/ta/llm.txt | 115 +++++++++--------- docs/i18n/te/llm.txt | 115 +++++++++--------- docs/i18n/th/llm.txt | 115 +++++++++--------- docs/i18n/tr/llm.txt | 115 +++++++++--------- docs/i18n/uk-UA/llm.txt | 115 +++++++++--------- docs/i18n/ur/llm.txt | 115 +++++++++--------- docs/i18n/vi/llm.txt | 115 +++++++++--------- docs/i18n/zh-CN/llm.txt | 115 +++++++++--------- docs/i18n/zh-TW/llm.txt | 115 +++++++++--------- llm.txt | 115 +++++++++--------- 45 files changed, 2379 insertions(+), 2582 deletions(-) create mode 100644 changelog.d/maintenance/root-cleanup-llmtxt-design-system.md rename design.md => docs/architecture/DESIGN_SYSTEM.md (92%) diff --git a/changelog.d/maintenance/root-cleanup-llmtxt-design-system.md b/changelog.d/maintenance/root-cleanup-llmtxt-design-system.md new file mode 100644 index 0000000000..acaad759f6 --- /dev/null +++ b/changelog.d/maintenance/root-cleanup-llmtxt-design-system.md @@ -0,0 +1 @@ +- **docs:** refresh `llm.txt` to the current project state (248 providers, 94 MCP tools / 30 scopes, 18 routing strategies, 12-factor Auto-Combo scoring, v3.8.47) and sync its 42 i18n mirrors; move the implemented design-system plan from the repo root to `docs/architecture/DESIGN_SYSTEM.md` rewritten as a reference doc. diff --git a/design.md b/docs/architecture/DESIGN_SYSTEM.md similarity index 92% rename from design.md rename to docs/architecture/DESIGN_SYSTEM.md index 64794e44a1..f68f1dd272 100644 --- a/design.md +++ b/docs/architecture/DESIGN_SYSTEM.md @@ -1,7 +1,18 @@ +--- +title: "Design System & Visual Identity" +lastUpdated: 2026-07-11 +--- + # OmniRoute — Design System & Visual Identity -> **Status:** standardization plan. **Phases 1–3 are implemented in this PR** (grid, primitives, status-color centralization, mono token, and the DataTable token migration). The DataTable migration is **faithful** — dark stays byte-identical (the new `--table-*` dark values equal the old hardcoded rgba); light is fixed (it was buggy always-dark via dead `var()` fallbacks). **⚠️ Wants a visual pass before merge** (light-theme tables + the secondary-text shift `#888`→`--color-text-muted`). **Phase 4 is now largely done too** (C6 focus-ring → accent, C7 Checkbox/Textarea primitives, C9 `cn()`→tailwind-merge); only the selective C8 hex-sweep remains. Note several remaining "hardcoded" hex are _intentional_ (always-dark console terminal, ReactFlow SVG strokes) and must NOT be swept. **Phase 5 (D4 + D8): the grid now reaches every standalone screen** (login/auth/error/legal/status/onboarding — their opaque `bg-bg` full-screen wrappers were hiding it) **and the dashboard content shell is fluid up to 4K** (`max-w-7xl` → `max-w-[3840px]`) so it follows the viewport on large monitors instead of centering with wide side gutters. **Phase 6 (D9): data tables are now opaque surfaces** so the grid no longer bleeds through their rows — card-less tables paint `bg-surface`, and the two log tables' semi-transparent `bg-black/5` tint (which tailwind-merge let win over the Card's `bg-surface`) is removed. The grid size itself is already correct (32px, identical to the site); a "bigger" grid on a running instance is a stale build, not code. -> **Date:** 2026-06-16 · **Scope:** unify the OmniRoute dashboard (`src/`) with the marketing site (`_mono_repo/omnirouteSite/`) into **one visual identity** — same graph-paper grid background, same color tokens, standardized components. +> **Status:** reference — the standardization described here is **implemented** (phases 1–6: grid wallpaper, primitives, status-color centralization, mono token, DataTable token migration, focus-ring → accent, Checkbox/Textarea primitives, `cn()` → tailwind-merge, grid on every standalone screen, fluid 4K content shell, opaque data-table surfaces). This document is the canonical description of the dashboard's design tokens, components, and conventions; the phase framing below is kept as the rationale for each decision. +> **Scope:** the OmniRoute dashboard (`src/`) and the marketing site (`_mono_repo/omnirouteSite/`) share **one visual identity** — same graph-paper grid background (32px), same color tokens, standardized components. +> +> Practical notes for maintainers: +> +> - Several remaining hardcoded hex values are **intentional** (always-dark console terminal, ReactFlow SVG strokes) and must **NOT** be swept into tokens. +> - A "bigger" grid on a running instance is a stale build, not code — the grid size is 32px, identical to the site. +> - Dark-theme `--table-*` values are byte-identical to the pre-migration hardcoded rgba; light theme was fixed (it was buggy always-dark via dead `var()` fallbacks). --- diff --git a/docs/i18n/ar/llm.txt b/docs/i18n/ar/llm.txt index 775cd2b66e..bbf5c27c78 100644 --- a/docs/i18n/ar/llm.txt +++ b/docs/i18n/ar/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/az/llm.txt b/docs/i18n/az/llm.txt index 45c98fe7db..f687ce262d 100644 --- a/docs/i18n/az/llm.txt +++ b/docs/i18n/az/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/bg/llm.txt b/docs/i18n/bg/llm.txt index 45c98fe7db..f687ce262d 100644 --- a/docs/i18n/bg/llm.txt +++ b/docs/i18n/bg/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/bn/llm.txt b/docs/i18n/bn/llm.txt index fdb0178a80..14269bdeaa 100644 --- a/docs/i18n/bn/llm.txt +++ b/docs/i18n/bn/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/cs/llm.txt b/docs/i18n/cs/llm.txt index baf7b823d8..d26dc026f3 100644 --- a/docs/i18n/cs/llm.txt +++ b/docs/i18n/cs/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/da/llm.txt b/docs/i18n/da/llm.txt index 819865a7ed..8c54b16c7f 100644 --- a/docs/i18n/da/llm.txt +++ b/docs/i18n/da/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/de/llm.txt b/docs/i18n/de/llm.txt index fd8b2bb12c..fc82d6b08e 100644 --- a/docs/i18n/de/llm.txt +++ b/docs/i18n/de/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/es/llm.txt b/docs/i18n/es/llm.txt index dc3ca451f0..fa6930d790 100644 --- a/docs/i18n/es/llm.txt +++ b/docs/i18n/es/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/fa/llm.txt b/docs/i18n/fa/llm.txt index 6848cd28f9..8758d5925b 100644 --- a/docs/i18n/fa/llm.txt +++ b/docs/i18n/fa/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/fi/llm.txt b/docs/i18n/fi/llm.txt index 96224f8a4d..7b832d7f64 100644 --- a/docs/i18n/fi/llm.txt +++ b/docs/i18n/fi/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/fr/llm.txt b/docs/i18n/fr/llm.txt index 7de5c69aac..7978e1e722 100644 --- a/docs/i18n/fr/llm.txt +++ b/docs/i18n/fr/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/gu/llm.txt b/docs/i18n/gu/llm.txt index 3123e0b0aa..716a3c1a50 100644 --- a/docs/i18n/gu/llm.txt +++ b/docs/i18n/gu/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/he/llm.txt b/docs/i18n/he/llm.txt index f73b2bb1cc..17466a0dce 100644 --- a/docs/i18n/he/llm.txt +++ b/docs/i18n/he/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/hi/llm.txt b/docs/i18n/hi/llm.txt index 64c9880884..45f2f5ee0e 100644 --- a/docs/i18n/hi/llm.txt +++ b/docs/i18n/hi/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/hu/llm.txt b/docs/i18n/hu/llm.txt index 9f574945c0..ba6d27cb24 100644 --- a/docs/i18n/hu/llm.txt +++ b/docs/i18n/hu/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/id/llm.txt b/docs/i18n/id/llm.txt index e05008dedb..281a63458b 100644 --- a/docs/i18n/id/llm.txt +++ b/docs/i18n/id/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/in/llm.txt b/docs/i18n/in/llm.txt index 6e91c869fd..3a9d45ef20 100644 --- a/docs/i18n/in/llm.txt +++ b/docs/i18n/in/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/it/llm.txt b/docs/i18n/it/llm.txt index 6db7b5f06a..c25fa077ba 100644 --- a/docs/i18n/it/llm.txt +++ b/docs/i18n/it/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/ja/llm.txt b/docs/i18n/ja/llm.txt index ab529087fe..0469265645 100644 --- a/docs/i18n/ja/llm.txt +++ b/docs/i18n/ja/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/ko/llm.txt b/docs/i18n/ko/llm.txt index 8c0aeb75c6..f33c181566 100644 --- a/docs/i18n/ko/llm.txt +++ b/docs/i18n/ko/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/mr/llm.txt b/docs/i18n/mr/llm.txt index cde497ddc3..f052ca9bd4 100644 --- a/docs/i18n/mr/llm.txt +++ b/docs/i18n/mr/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/ms/llm.txt b/docs/i18n/ms/llm.txt index daf937c1f3..8850fb0de7 100644 --- a/docs/i18n/ms/llm.txt +++ b/docs/i18n/ms/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/nl/llm.txt b/docs/i18n/nl/llm.txt index 38c2c89661..6465050541 100644 --- a/docs/i18n/nl/llm.txt +++ b/docs/i18n/nl/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/no/llm.txt b/docs/i18n/no/llm.txt index 1d19c59c96..be667f94f0 100644 --- a/docs/i18n/no/llm.txt +++ b/docs/i18n/no/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/phi/llm.txt b/docs/i18n/phi/llm.txt index 783034fdfb..fae6a95de1 100644 --- a/docs/i18n/phi/llm.txt +++ b/docs/i18n/phi/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/pl/llm.txt b/docs/i18n/pl/llm.txt index 1072ad9f24..50a20e3320 100644 --- a/docs/i18n/pl/llm.txt +++ b/docs/i18n/pl/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/pt-BR/llm.txt b/docs/i18n/pt-BR/llm.txt index c117a31086..b40da47c95 100644 --- a/docs/i18n/pt-BR/llm.txt +++ b/docs/i18n/pt-BR/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/pt/llm.txt b/docs/i18n/pt/llm.txt index 0624b72f3b..9568237c5d 100644 --- a/docs/i18n/pt/llm.txt +++ b/docs/i18n/pt/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/ro/llm.txt b/docs/i18n/ro/llm.txt index 2810dae83d..f92f0b2444 100644 --- a/docs/i18n/ro/llm.txt +++ b/docs/i18n/ro/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/ru/llm.txt b/docs/i18n/ru/llm.txt index 923ed9e785..ccd8bcecc4 100644 --- a/docs/i18n/ru/llm.txt +++ b/docs/i18n/ru/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/sk/llm.txt b/docs/i18n/sk/llm.txt index bf2646ae46..e19bf8803f 100644 --- a/docs/i18n/sk/llm.txt +++ b/docs/i18n/sk/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/sv/llm.txt b/docs/i18n/sv/llm.txt index 9415c9010d..2d7cc63ed2 100644 --- a/docs/i18n/sv/llm.txt +++ b/docs/i18n/sv/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/sw/llm.txt b/docs/i18n/sw/llm.txt index f9ebdc2a3d..fc08f7b142 100644 --- a/docs/i18n/sw/llm.txt +++ b/docs/i18n/sw/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/ta/llm.txt b/docs/i18n/ta/llm.txt index 91f2022328..1015b8850a 100644 --- a/docs/i18n/ta/llm.txt +++ b/docs/i18n/ta/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/te/llm.txt b/docs/i18n/te/llm.txt index 1de33d1956..375192f15a 100644 --- a/docs/i18n/te/llm.txt +++ b/docs/i18n/te/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/th/llm.txt b/docs/i18n/th/llm.txt index cd0a37688c..64e8b57d81 100644 --- a/docs/i18n/th/llm.txt +++ b/docs/i18n/th/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/tr/llm.txt b/docs/i18n/tr/llm.txt index 634d4bc259..74ef1375db 100644 --- a/docs/i18n/tr/llm.txt +++ b/docs/i18n/tr/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/uk-UA/llm.txt b/docs/i18n/uk-UA/llm.txt index 14cf3b6600..54e0cafbc0 100644 --- a/docs/i18n/uk-UA/llm.txt +++ b/docs/i18n/uk-UA/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/ur/llm.txt b/docs/i18n/ur/llm.txt index c402cff2a1..105c5cd699 100644 --- a/docs/i18n/ur/llm.txt +++ b/docs/i18n/ur/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/vi/llm.txt b/docs/i18n/vi/llm.txt index e21816296c..24c03f7a2d 100644 --- a/docs/i18n/vi/llm.txt +++ b/docs/i18n/vi/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/zh-CN/llm.txt b/docs/i18n/zh-CN/llm.txt index 090969a4f0..4a21c4fe0d 100644 --- a/docs/i18n/zh-CN/llm.txt +++ b/docs/i18n/zh-CN/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/zh-TW/llm.txt b/docs/i18n/zh-TW/llm.txt index 891f389de1..0c39f612a5 100644 --- a/docs/i18n/zh-TW/llm.txt +++ b/docs/i18n/zh-TW/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/llm.txt b/llm.txt index d2995c50a4..9e012ae34d 100644 --- a/llm.txt +++ b/llm.txt @@ -1,6 +1,6 @@ # OmniRoute -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -8,12 +8,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -41,7 +41,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -102,7 +102,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -124,7 +124,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -165,7 +165,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -182,7 +182,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -208,15 +208,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -224,7 +224,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -253,24 +253,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -278,15 +275,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -299,7 +296,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -315,7 +312,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -345,18 +342,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -381,17 +381,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -435,15 +435,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -474,21 +474,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links