diff --git a/changelog.d/features/6776-discover-live-codex-models.md b/changelog.d/features/6776-discover-live-codex-models.md new file mode 100644 index 0000000000..111b674056 --- /dev/null +++ b/changelog.d/features/6776-discover-live-codex-models.md @@ -0,0 +1 @@ +- **feat(codex):** Codex provider model discovery now fetches the live catalog from `chatgpt.com/backend-api/codex/models` using Codex-shaped headers, falling back to a GitHub-hosted model manifest and then to the local static catalog when the live/GitHub sources are unavailable or return an unexpected shape — new `src/app/api/providers/[id]/models/discovery/codex.ts` (normalization, version-gating, merge/enrich against the local catalog) covered by `tests/unit/provider-models-discovery-split.test.ts` and `tests/unit/provider-models-route-codex.test.ts` (#6776 — thanks @JxnLexn). diff --git a/changelog.d/fixes/6723-i18n-pt-br-backfill.md b/changelog.d/fixes/6723-i18n-pt-br-backfill.md new file mode 100644 index 0000000000..aca0c58e24 --- /dev/null +++ b/changelog.d/fixes/6723-i18n-pt-br-backfill.md @@ -0,0 +1 @@ +- **fix(i18n):** pt-BR was missing 194 UI keys present in `en.json` — a real, silent data-sync gap, not covered by any duplicate/mislabeled #6694 (that issue's 9 `providers.*` keys are disjoint, present-but-untranslated sentinels caused by a separate `providerText()` fallback bug) ([#6695](https://github.com/diegosouzapw/OmniRoute/issues/6695)) — `scripts/i18n/sync-ui-keys.mjs` (which mirrors newly-added `en.json` keys into every locale) wasn't re-run after recent `en.json` additions, and the CI `i18n:check-ui-coverage` gate only fails a locale below an 80% threshold, so pt-BR stayed green at 93.8% coverage despite the gap. Backfilled all 194 missing keys into `src/i18n/messages/pt-BR.json` (translated to Brazilian Portuguese, no leftover `__MISSING__` markers) via `npm run i18n:sync-ui -- --locale=pt-BR` + manual translation. Regression guard: `tests/unit/i18n-pt-br.test.ts` (new case asserting full `en.json`→`pt-BR.json` key parity, so a future drift fails a fast unit test instead of silently degrading the coverage percentage). diff --git a/changelog.d/maintenance/root-cleanup-llmtxt-design-system.md b/changelog.d/maintenance/root-cleanup-llmtxt-design-system.md new file mode 100644 index 0000000000..acaad759f6 --- /dev/null +++ b/changelog.d/maintenance/root-cleanup-llmtxt-design-system.md @@ -0,0 +1 @@ +- **docs:** refresh `llm.txt` to the current project state (248 providers, 94 MCP tools / 30 scopes, 18 routing strategies, 12-factor Auto-Combo scoring, v3.8.47) and sync its 42 i18n mirrors; move the implemented design-system plan from the repo root to `docs/architecture/DESIGN_SYSTEM.md` rewritten as a reference doc. diff --git a/config/quality/eslint-suppressions.json b/config/quality/eslint-suppressions.json index 9065f5db48..2571ec8f3d 100644 --- a/config/quality/eslint-suppressions.json +++ b/config/quality/eslint-suppressions.json @@ -1774,7 +1774,7 @@ }, "tests/unit/provider-models-route.test.ts": { "@typescript-eslint/no-explicit-any": { - "count": 57 + "count": 55 } }, "tests/unit/provider-models-token-limits.test.ts": { diff --git a/design.md b/docs/architecture/DESIGN_SYSTEM.md similarity index 92% rename from design.md rename to docs/architecture/DESIGN_SYSTEM.md index 64794e44a1..f68f1dd272 100644 --- a/design.md +++ b/docs/architecture/DESIGN_SYSTEM.md @@ -1,7 +1,18 @@ +--- +title: "Design System & Visual Identity" +lastUpdated: 2026-07-11 +--- + # OmniRoute — Design System & Visual Identity -> **Status:** standardization plan. **Phases 1–3 are implemented in this PR** (grid, primitives, status-color centralization, mono token, and the DataTable token migration). The DataTable migration is **faithful** — dark stays byte-identical (the new `--table-*` dark values equal the old hardcoded rgba); light is fixed (it was buggy always-dark via dead `var()` fallbacks). **⚠️ Wants a visual pass before merge** (light-theme tables + the secondary-text shift `#888`→`--color-text-muted`). **Phase 4 is now largely done too** (C6 focus-ring → accent, C7 Checkbox/Textarea primitives, C9 `cn()`→tailwind-merge); only the selective C8 hex-sweep remains. Note several remaining "hardcoded" hex are _intentional_ (always-dark console terminal, ReactFlow SVG strokes) and must NOT be swept. **Phase 5 (D4 + D8): the grid now reaches every standalone screen** (login/auth/error/legal/status/onboarding — their opaque `bg-bg` full-screen wrappers were hiding it) **and the dashboard content shell is fluid up to 4K** (`max-w-7xl` → `max-w-[3840px]`) so it follows the viewport on large monitors instead of centering with wide side gutters. **Phase 6 (D9): data tables are now opaque surfaces** so the grid no longer bleeds through their rows — card-less tables paint `bg-surface`, and the two log tables' semi-transparent `bg-black/5` tint (which tailwind-merge let win over the Card's `bg-surface`) is removed. The grid size itself is already correct (32px, identical to the site); a "bigger" grid on a running instance is a stale build, not code. -> **Date:** 2026-06-16 · **Scope:** unify the OmniRoute dashboard (`src/`) with the marketing site (`_mono_repo/omnirouteSite/`) into **one visual identity** — same graph-paper grid background, same color tokens, standardized components. +> **Status:** reference — the standardization described here is **implemented** (phases 1–6: grid wallpaper, primitives, status-color centralization, mono token, DataTable token migration, focus-ring → accent, Checkbox/Textarea primitives, `cn()` → tailwind-merge, grid on every standalone screen, fluid 4K content shell, opaque data-table surfaces). This document is the canonical description of the dashboard's design tokens, components, and conventions; the phase framing below is kept as the rationale for each decision. +> **Scope:** the OmniRoute dashboard (`src/`) and the marketing site (`_mono_repo/omnirouteSite/`) share **one visual identity** — same graph-paper grid background (32px), same color tokens, standardized components. +> +> Practical notes for maintainers: +> +> - Several remaining hardcoded hex values are **intentional** (always-dark console terminal, ReactFlow SVG strokes) and must **NOT** be swept into tokens. +> - A "bigger" grid on a running instance is a stale build, not code — the grid size is 32px, identical to the site. +> - Dark-theme `--table-*` values are byte-identical to the pre-migration hardcoded rgba; light theme was fixed (it was buggy always-dark via dead `var()` fallbacks). --- diff --git a/docs/i18n/ar/llm.txt b/docs/i18n/ar/llm.txt index 775cd2b66e..bbf5c27c78 100644 --- a/docs/i18n/ar/llm.txt +++ b/docs/i18n/ar/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/az/llm.txt b/docs/i18n/az/llm.txt index 45c98fe7db..f687ce262d 100644 --- a/docs/i18n/az/llm.txt +++ b/docs/i18n/az/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/bg/llm.txt b/docs/i18n/bg/llm.txt index 45c98fe7db..f687ce262d 100644 --- a/docs/i18n/bg/llm.txt +++ b/docs/i18n/bg/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/bn/llm.txt b/docs/i18n/bn/llm.txt index fdb0178a80..14269bdeaa 100644 --- a/docs/i18n/bn/llm.txt +++ b/docs/i18n/bn/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/cs/llm.txt b/docs/i18n/cs/llm.txt index baf7b823d8..d26dc026f3 100644 --- a/docs/i18n/cs/llm.txt +++ b/docs/i18n/cs/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/da/llm.txt b/docs/i18n/da/llm.txt index 819865a7ed..8c54b16c7f 100644 --- a/docs/i18n/da/llm.txt +++ b/docs/i18n/da/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/de/llm.txt b/docs/i18n/de/llm.txt index fd8b2bb12c..fc82d6b08e 100644 --- a/docs/i18n/de/llm.txt +++ b/docs/i18n/de/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/es/llm.txt b/docs/i18n/es/llm.txt index dc3ca451f0..fa6930d790 100644 --- a/docs/i18n/es/llm.txt +++ b/docs/i18n/es/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/fa/llm.txt b/docs/i18n/fa/llm.txt index 6848cd28f9..8758d5925b 100644 --- a/docs/i18n/fa/llm.txt +++ b/docs/i18n/fa/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/fi/llm.txt b/docs/i18n/fi/llm.txt index 96224f8a4d..7b832d7f64 100644 --- a/docs/i18n/fi/llm.txt +++ b/docs/i18n/fi/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/fr/llm.txt b/docs/i18n/fr/llm.txt index 7de5c69aac..7978e1e722 100644 --- a/docs/i18n/fr/llm.txt +++ b/docs/i18n/fr/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/gu/llm.txt b/docs/i18n/gu/llm.txt index 3123e0b0aa..716a3c1a50 100644 --- a/docs/i18n/gu/llm.txt +++ b/docs/i18n/gu/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/he/llm.txt b/docs/i18n/he/llm.txt index f73b2bb1cc..17466a0dce 100644 --- a/docs/i18n/he/llm.txt +++ b/docs/i18n/he/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/hi/llm.txt b/docs/i18n/hi/llm.txt index 64c9880884..45f2f5ee0e 100644 --- a/docs/i18n/hi/llm.txt +++ b/docs/i18n/hi/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/hu/llm.txt b/docs/i18n/hu/llm.txt index 9f574945c0..ba6d27cb24 100644 --- a/docs/i18n/hu/llm.txt +++ b/docs/i18n/hu/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/id/llm.txt b/docs/i18n/id/llm.txt index e05008dedb..281a63458b 100644 --- a/docs/i18n/id/llm.txt +++ b/docs/i18n/id/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/in/llm.txt b/docs/i18n/in/llm.txt index 6e91c869fd..3a9d45ef20 100644 --- a/docs/i18n/in/llm.txt +++ b/docs/i18n/in/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/it/llm.txt b/docs/i18n/it/llm.txt index 6db7b5f06a..c25fa077ba 100644 --- a/docs/i18n/it/llm.txt +++ b/docs/i18n/it/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/ja/llm.txt b/docs/i18n/ja/llm.txt index ab529087fe..0469265645 100644 --- a/docs/i18n/ja/llm.txt +++ b/docs/i18n/ja/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/ko/llm.txt b/docs/i18n/ko/llm.txt index 8c0aeb75c6..f33c181566 100644 --- a/docs/i18n/ko/llm.txt +++ b/docs/i18n/ko/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/mr/llm.txt b/docs/i18n/mr/llm.txt index cde497ddc3..f052ca9bd4 100644 --- a/docs/i18n/mr/llm.txt +++ b/docs/i18n/mr/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/ms/llm.txt b/docs/i18n/ms/llm.txt index daf937c1f3..8850fb0de7 100644 --- a/docs/i18n/ms/llm.txt +++ b/docs/i18n/ms/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/nl/llm.txt b/docs/i18n/nl/llm.txt index 38c2c89661..6465050541 100644 --- a/docs/i18n/nl/llm.txt +++ b/docs/i18n/nl/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/no/llm.txt b/docs/i18n/no/llm.txt index 1d19c59c96..be667f94f0 100644 --- a/docs/i18n/no/llm.txt +++ b/docs/i18n/no/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/phi/llm.txt b/docs/i18n/phi/llm.txt index 783034fdfb..fae6a95de1 100644 --- a/docs/i18n/phi/llm.txt +++ b/docs/i18n/phi/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/pl/llm.txt b/docs/i18n/pl/llm.txt index 1072ad9f24..50a20e3320 100644 --- a/docs/i18n/pl/llm.txt +++ b/docs/i18n/pl/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/pt-BR/llm.txt b/docs/i18n/pt-BR/llm.txt index c117a31086..b40da47c95 100644 --- a/docs/i18n/pt-BR/llm.txt +++ b/docs/i18n/pt-BR/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/pt/llm.txt b/docs/i18n/pt/llm.txt index 0624b72f3b..9568237c5d 100644 --- a/docs/i18n/pt/llm.txt +++ b/docs/i18n/pt/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/ro/llm.txt b/docs/i18n/ro/llm.txt index 2810dae83d..f92f0b2444 100644 --- a/docs/i18n/ro/llm.txt +++ b/docs/i18n/ro/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/ru/llm.txt b/docs/i18n/ru/llm.txt index 923ed9e785..ccd8bcecc4 100644 --- a/docs/i18n/ru/llm.txt +++ b/docs/i18n/ru/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/sk/llm.txt b/docs/i18n/sk/llm.txt index bf2646ae46..e19bf8803f 100644 --- a/docs/i18n/sk/llm.txt +++ b/docs/i18n/sk/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/sv/llm.txt b/docs/i18n/sv/llm.txt index 9415c9010d..2d7cc63ed2 100644 --- a/docs/i18n/sv/llm.txt +++ b/docs/i18n/sv/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/sw/llm.txt b/docs/i18n/sw/llm.txt index f9ebdc2a3d..fc08f7b142 100644 --- a/docs/i18n/sw/llm.txt +++ b/docs/i18n/sw/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/ta/llm.txt b/docs/i18n/ta/llm.txt index 91f2022328..1015b8850a 100644 --- a/docs/i18n/ta/llm.txt +++ b/docs/i18n/ta/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/te/llm.txt b/docs/i18n/te/llm.txt index 1de33d1956..375192f15a 100644 --- a/docs/i18n/te/llm.txt +++ b/docs/i18n/te/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/th/llm.txt b/docs/i18n/th/llm.txt index cd0a37688c..64e8b57d81 100644 --- a/docs/i18n/th/llm.txt +++ b/docs/i18n/th/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/tr/llm.txt b/docs/i18n/tr/llm.txt index 634d4bc259..74ef1375db 100644 --- a/docs/i18n/tr/llm.txt +++ b/docs/i18n/tr/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/uk-UA/llm.txt b/docs/i18n/uk-UA/llm.txt index 14cf3b6600..54e0cafbc0 100644 --- a/docs/i18n/uk-UA/llm.txt +++ b/docs/i18n/uk-UA/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/ur/llm.txt b/docs/i18n/ur/llm.txt index c402cff2a1..105c5cd699 100644 --- a/docs/i18n/ur/llm.txt +++ b/docs/i18n/ur/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/vi/llm.txt b/docs/i18n/vi/llm.txt index e21816296c..24c03f7a2d 100644 --- a/docs/i18n/vi/llm.txt +++ b/docs/i18n/vi/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/zh-CN/llm.txt b/docs/i18n/zh-CN/llm.txt index 090969a4f0..4a21c4fe0d 100644 --- a/docs/i18n/zh-CN/llm.txt +++ b/docs/i18n/zh-CN/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/docs/i18n/zh-TW/llm.txt b/docs/i18n/zh-TW/llm.txt index 891f389de1..0c39f612a5 100644 --- a/docs/i18n/zh-TW/llm.txt +++ b/docs/i18n/zh-TW/llm.txt @@ -4,7 +4,7 @@ --- -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -12,12 +12,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -45,7 +45,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -106,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -128,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -169,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -186,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -212,15 +212,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -228,7 +228,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -257,24 +257,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -282,15 +279,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -303,7 +300,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -319,7 +316,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -349,18 +346,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -385,17 +385,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -439,15 +439,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -478,21 +478,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/llm.txt b/llm.txt index d2995c50a4..9e012ae34d 100644 --- a/llm.txt +++ b/llm.txt @@ -1,6 +1,6 @@ # OmniRoute -> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 177 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (37 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. +> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 248 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (94 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex-cloud, devin, jules), Guardrails framework, and an Electron desktop app. ## Overview @@ -8,12 +8,12 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo **Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost. -**Current version:** 3.8.8 +**Current version:** 3.8.47 ## Tech Stack -- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) -- **Framework:** Next.js 16 (App Router) with TypeScript 5.9 +- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`) +- **Framework:** Next.js 16 (App Router) with TypeScript 6 - **Database:** SQLite via better-sqlite3 (local, zero-config, 110+ migrations) - **State management:** Zustand (client), SQLite (server persistence) - **UI:** React 19, Tailwind CSS 4, Recharts for analytics, @lobehub/icons for 130+ provider SVG icons @@ -41,7 +41,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── auto-combo/ # Auto-combo engine dashboard │ │ │ ├── cache/ # Cache dashboard (semantic cache stats) │ │ │ ├── cli-tools/ # CLI tool configuration (Claude Code, Codex, etc.) -│ │ │ ├── combos/ # Model combo management (14 strategies + 4 templates) +│ │ │ ├── combos/ # Model combo management (18 strategies + 4 templates) │ │ │ ├── costs/ # Cost tracking per provider/model │ │ │ ├── endpoint/ # Unified: Endpoint Proxy, MCP, A2A, API Endpoints tabs │ │ │ ├── health/ # System health (uptime, circuit breakers, latency) @@ -102,7 +102,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ └── streaming.ts # SSE streaming for A2A │ │ ├── acp/ # Agent Communication Protocol registry and manager │ │ ├── compliance/ # Compliance policy engine -│ │ ├── db/ # SQLite database layer (95+ modules + migrations) +│ │ ├── db/ # SQLite database layer (99 modules + migrations) │ │ │ ├── core.ts # Database initialization, connection, schema │ │ │ ├── providers.ts # Provider connection CRUD │ │ │ ├── models.ts # Model catalog management @@ -124,7 +124,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ │ ├── secrets.ts # Secrets management │ │ │ ├── stateReset.ts # State reset utilities │ │ │ ├── migrationRunner.ts # Schema migration runner -│ │ │ └── migrations/ # 110+ versioned SQL migration files +│ │ │ └── migrations/ # 117 versioned SQL migration files │ │ ├── evals/ # Eval runner and scheduler │ │ ├── memory/ # Persistent conversational memory │ │ │ ├── extraction.ts # Memory extraction from conversations @@ -165,7 +165,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ └── manager.ts # MITM proxy manager │ ├── shared/ # Shared utilities, components, and constants │ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.) -│ │ ├── constants/ # Provider definitions (160+), model lists, pricing, routing strategies, MCP scopes +│ │ ├── constants/ # Provider definitions (248), model lists, pricing, routing strategies, MCP scopes │ │ ├── contracts/ # Shared API contracts │ │ ├── hooks/ # React hooks │ │ ├── middleware/ # Shared middleware utilities @@ -182,7 +182,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ├── open-sse/ # Standalone SSE server (npm workspace) │ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video, │ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models) -│ ├── executors/ # Provider-specific request executors (31 executors) +│ ├── executors/ # Provider-specific request executors (78 executor modules) │ │ ├── base.ts # Base executor with shared logic │ │ ├── default.ts # Default OpenAI-compatible executor │ │ ├── cursor.ts # Cursor IDE (protobuf + checksum) @@ -208,15 +208,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── moderations.ts # Content moderation │ │ ├── rerank.ts # Reranking API │ │ └── search.ts # Web search API -│ ├── mcp-server/ # Built-in MCP server (29 tools, 3 transports: stdio/SSE/streamable-HTTP) +│ ├── mcp-server/ # Built-in MCP server (94 tools, 3 transports: stdio/SSE/streamable-HTTP) │ │ ├── server.ts # MCP server core (tool registration, scope enforcement) │ │ ├── tools/ # Tool implementations (advancedTools, memoryTools, skillTools) │ │ ├── schemas/ # Zod input schemas (tools, audit, a2a) -│ │ ├── scopeEnforcement.ts # Scope-based access control (10 scopes) +│ │ ├── scopeEnforcement.ts # Scope-based access control (30 scopes) │ │ ├── audit.ts # Tool call audit logging │ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat │ │ └── httpTransport.ts # HTTP transport handler -│ ├── services/ # 36+ service modules +│ ├── services/ # 140+ service modules │ │ ├── combo.ts # Core routing engine │ │ ├── usage.ts # Usage tracking │ │ ├── tokenRefresh.ts # OAuth token refresh @@ -224,7 +224,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ │ ├── accountFallback.ts # Multi-account fallback │ │ ├── sessionManager.ts # Session management │ │ ├── wildcardRouter.ts # Wildcard model routing -│ │ ├── autoCombo/ # Auto-combo engine (6-factor scoring, bandit exploration) +│ │ ├── autoCombo/ # Auto-combo engine (12-factor scoring, bandit exploration) │ │ ├── intentClassifier.ts # Request intent classification │ │ ├── taskAwareRouter.ts # Task-aware routing │ │ ├── thinkingBudget.ts # Thinking budget management @@ -253,24 +253,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo │ ├── preload.js # Preload script (IPC bridge) │ └── assets/ # App icons and assets ├── tests/ # Test suites -│ ├── unit/ # 122 unit test files +│ ├── unit/ # 2,700+ unit test files │ ├── integration/ # Integration tests │ ├── e2e/ # Playwright E2E tests │ ├── security/ # Security tests │ ├── translator/ # Translator-specific tests │ └── load/ # Load tests ├── docs/ # Documentation -│ ├── i18n/ # 30-language translated docs -│ ├── ARCHITECTURE.md # Full architecture documentation -│ ├── API_REFERENCE.md # API reference -│ ├── USER_GUIDE.md # User guide -│ ├── CODEBASE_DOCUMENTATION.md # Codebase overview -│ ├── CLI-TOOLS.md # CLI tools integration guide -│ ├── A2A-SERVER.md # A2A agent protocol documentation -│ ├── AUTO-COMBO.md # Auto-combo engine (6-factor scoring) -│ ├── MCP-SERVER.md # MCP server (29 tools) -│ ├── TROUBLESHOOTING.md # Troubleshooting guide -│ ├── VM_DEPLOYMENT_GUIDE.md # VPS deployment guide +│ ├── i18n/ # 43-language translated docs +│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md +│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md +│ ├── frameworks/ # MCP-SERVER.md (94 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md +│ ├── routing/ # AUTO-COMBO.md (12-factor scoring), REASONING_REPLAY.md +│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md +│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md +│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment │ ├── openapi.yaml # OpenAPI specification │ └── screenshots/ # Dashboard screenshots ├── bin/ # CLI entry points (omniroute, reset-password) @@ -278,15 +275,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo └── .env.example # Environment variable template ``` -## Key Features (v3.8.8) +## Key Features (v3.8.47) ### Core Proxy -- **177 AI providers** with automatic format translation -- **4 provider categories**: Free (5), OAuth (14), API Key (123+), Self-Hosted (8+), Custom (OpenAI/Anthropic-compatible) -- **14 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay, **reset-aware** (v3.8) +- **248 AI providers** with automatic format translation +- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible) +- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline - **4-tier fallback**: Subscription → API Key → Cheap → Free - **Context Relay strategy**: Session handoff summaries on account rotation for continuity -- **Auto-combo engine**: Self-healing routing optimization with **9-factor scoring** (health/quota/costInv/latencyInv/taskFit/specificityMatch/stability/tierPriority/tierAffinity), bandit exploration, progressive cooldown +- **Auto-combo engine**: Self-healing routing optimization with **12-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown - **Semantic caching** with cache hit/miss headers - **Idempotency** with configurable dedup window - **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout @@ -299,7 +296,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered) - **MITM Proxy**: Certificate management, DNS handling, and target routing - **Cloudflare Tunnels**: Managed tunnel creation for remote access -- **Coverage gate**: 75% statements/lines/functions, 70% branches (measured ~82%) +- **Coverage gate**: ratchet vs `quality-baseline.json`; absolute floor 60% statements/lines/functions/branches ### Security - **Data Loss Prevention**: SQLite migration safety bounds abort startup on dangerous massive schema overrides. Pre-migration `VACUUM INTO` backups isolate rollback snapshots. @@ -315,7 +312,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo ### Dashboard Pages (23 sections) - **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons -- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 14 strategies +- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 18 strategies - **Auto-Combo** — Auto-combo engine dashboard with scoring metrics - **Analytics** — Token consumption, cost, heatmaps, distributions - **Health** — Uptime, memory, latency percentiles, circuit breakers @@ -345,18 +342,21 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}` - **Ollama** — `/v1/api/chat`, `/api/tags` - **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily) -- **MCP** — 37-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) +- **MCP** — 94-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP) - **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 5 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report) - **ACP** — Agent Communication Protocol registry and manager -### MCP Server (37 Tools) -| Category | Tools | -|------------|-------| -| Core (30) | `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`, `web_search`, `simulate_route`, `set_budget_guard`, `set_routing_strategy`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`, `db_health_check`, `sync_pricing`, `cache_stats`, `cache_flush`, and advanced routing/diagnostics tools (see `docs/frameworks/MCP-SERVER.md` for full inventory) | -| Memory (3) | `memory_search`, `memory_add`, `memory_clear` | -| Skills (4) | `skills_list`, `skills_enable`, `skills_execute`, `skills_executions` | +### MCP Server (94 Tools) -**MCP Auth Scopes (~13):** `read:health`, `read:combos`, `write:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, `write:resilience`, plus memory/skills scopes — full list in `docs/frameworks/MCP-SERVER.md`. +94 tools across modules: **34 base** (health, combos, quotas, routing, cost, models, cache, +diagnostics) plus **memory**, **skill**, **agentSkill**, **pool**, **notion**, **obsidian**, +**gamification**, and **plugin** modules. Full per-tool inventory: +`docs/frameworks/MCP-SERVER.md`. + +**MCP Auth Scopes (30):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`, +`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`, +`write:resilience`, plus memory/skills/pool/plugin scopes — full list in +`docs/frameworks/MCP-SERVER.md`. ### Provider Categories @@ -381,17 +381,17 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 3. **Connection-based provider model:** Providers are stored as "connections" in SQLite. Each connection has an `id`, `provider`, `authType` (oauth/apikey/free), `isActive` flag, and credentials. Multiple connections per provider for multi-account rotation. -4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 14 strategies including auto-combo with self-healing and context-relay for session continuity. +4. **Combo system for fallback:** Users create "combos" — ordered lists of `provider/model` pairs. The proxy tries each in order until one succeeds. Supports 18 strategies including auto-combo with self-healing and context-relay for session continuity. 5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client. -6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 21 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. +6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes. 7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`). 8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages. -9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 21 `src/lib/db/` modules with 16 SQL migrations. +9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 99 `src/lib/db/` modules with 117 SQL migrations. 10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`. @@ -435,15 +435,15 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo 4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`. -5. **Database layer:** Operations go through `src/lib/db/` modules (95+ domain-specific files, 110+ migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. +5. **Database layer:** Operations go through `src/lib/db/` modules (99 domain-specific files, 117 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module. -6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: 75% statements/lines/functions, 70% branches. +6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches. 7. **MCP and A2A pages are embedded as tabs inside `/dashboard/endpoint`**, not standalone routes. 8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB. -9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **9-factor scoring** (health 0.22, quota 0.17, costInv 0.17, latencyInv 0.13, taskFit 0.08, specificityMatch 0.08, stability 0.05, tierPriority 0.05, tierAffinity 0.05), 4 mode packs, bandit exploration, progressive cooldown. +9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **12-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown. 10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130). @@ -474,21 +474,16 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo - **Connection Cooldown** (`src/sse/services/auth.ts::markAccountUnavailable`) — one key/account scope. - **Model Lockout** (`open-sse/services/accountFallback.ts`) — provider + connection + model scope. -## v3.8.0 Highlights +## v3.8.x Highlights -- **Cloud Agents** (Codex Cloud, Devin, Jules) with task lifecycle management and management-auth enforcement -- **Guardrails framework**: hot-reloadable registry with vision-bridge, pii-masker, prompt-injection -- **9-factor Auto-Combo scoring** (was 6-factor in earlier versions) -- **`reset-aware` routing strategy** (14th strategy) — picks the account whose quota will reset soonest -- **A2A protocol expanded to 5 skills**: smart-routing, quota-management, provider-discovery, cost-analysis, health-report -- **MCP server expanded to 37 tools** (30 base + 3 memory + 4 skills) across ~13 scopes -- **OAuth providers expanded to 14**: added Qwen, Kiro, Qoder, Gemini, Windsurf, GitLab Duo -- **Coverage gate raised to 75/75/75/70** (was 60% across the board) — measured ~82% -- **Reasoning replay** (`docs/routing/REASONING_REPLAY.md`) — capture and inspect provider reasoning streams -- **Compliance + Evals + Webhooks** documentation introduced -- **Stealth guide** (`docs/security/STEALTH_GUIDE.md`) — TLS / CLI fingerprint configuration -- **Tunnels guide** (`docs/ops/TUNNELS_GUIDE.md`) — Cloudflare tunnel management -- **Electron guide** (`docs/guides/ELECTRON_GUIDE.md`) — desktop app build + signing +- **248-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add +- **18 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay` +- **12-factor Auto-Combo scoring** with bandit exploration and progressive cooldown +- **MCP server expanded to 94 tools / 30 scopes** (base + memory/skill/agentSkill/pool/notion/obsidian/gamification/plugin modules) +- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks +- **Embedded services** manager (install/start/stop bundled services from the dashboard) +- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic +- Full changelog: `CHANGELOG.md` ## Links diff --git a/open-sse/handlers/chatCore.ts b/open-sse/handlers/chatCore.ts index 99fdd35267..c9b742843d 100644 --- a/open-sse/handlers/chatCore.ts +++ b/open-sse/handlers/chatCore.ts @@ -297,6 +297,7 @@ import { import { resolveBackgroundTaskRedirect } from "./chatCore/backgroundRedirect.ts"; import type { CompressionConfig, CompressionPipelineStep } from "../services/compression/types.ts"; import { prepareWebSearchFallbackBody } from "../services/webSearchFallback.ts"; +import { resolveInterceptSearch } from "@/lib/db/interceptionRules"; import { resolveExplicitStreamAlias, resolveStreamFlag, @@ -732,12 +733,17 @@ export async function handleChatCore({ // Initialize rate limit settings from persisted DB (once, lazy) await initializeRateLimits(); + // #3384: per-model interception rule (src/lib/db/interceptionRules.ts) overrides the + // native-bypass defaults below when the operator explicitly configured it for this + // provider/model pair; undefined falls through to the existing bypass logic. + const interceptSearchOverride = resolveInterceptSearch(provider, effectiveModel); const { body: bodyWithWebSearchFallback, fallback: webSearchFallbackPlan } = prepareWebSearchFallbackBody(body as Record, { provider, sourceFormat, targetFormat, nativeCodexPassthrough, + interceptSearchOverride, }); if (webSearchFallbackPlan.enabled) { body = bodyWithWebSearchFallback as typeof body; diff --git a/open-sse/services/webSearchFallback.ts b/open-sse/services/webSearchFallback.ts index f82b565630..9d80610d47 100644 --- a/open-sse/services/webSearchFallback.ts +++ b/open-sse/services/webSearchFallback.ts @@ -142,12 +142,20 @@ export function supportsNativeWebSearchFallbackBypass({ sourceFormat, targetFormat, nativeCodexPassthrough, + interceptSearchOverride, }: { provider?: string | null; sourceFormat?: string | null; targetFormat: string | null | undefined; nativeCodexPassthrough: boolean; + // Per-model rule (#3384) — resolveInterceptSearch() in src/lib/db/interceptionRules.ts. + // true = force interception (never bypass); false = force native bypass; undefined = + // fall through to the native-bypass defaults below. + interceptSearchOverride?: boolean; }): boolean { + if (typeof interceptSearchOverride === "boolean") { + return !interceptSearchOverride; + } // Native Codex (OpenAI Responses) passthrough: the upstream runs web search itself. if (nativeCodexPassthrough) return true; // Gemini target: the Gemini translator maps built-in web search to googleSearch natively. @@ -171,6 +179,7 @@ export function prepareWebSearchFallbackBody( sourceFormat?: string | null; targetFormat?: string | null; nativeCodexPassthrough: boolean; + interceptSearchOverride?: boolean; } ): { body: T; fallback: WebSearchFallbackPlan } { const tools = Array.isArray(body.tools) ? body.tools : null; diff --git a/src/app/(dashboard)/dashboard/providers/[id]/ProviderDetailPageClient.tsx b/src/app/(dashboard)/dashboard/providers/[id]/ProviderDetailPageClient.tsx index 87cfc2bc0e..0862282ba2 100644 --- a/src/app/(dashboard)/dashboard/providers/[id]/ProviderDetailPageClient.tsx +++ b/src/app/(dashboard)/dashboard/providers/[id]/ProviderDetailPageClient.tsx @@ -83,10 +83,7 @@ export default function ProviderDetailPageClient() { isAnthropicCompatibleProvider(providerId) && !isClaudeCodeCompatibleProvider(providerId); const isCompatible = isOpenAICompatible || isAnthropicCompatible || isCcCompatible; const isAnthropicProtocolCompatible = isAnthropicCompatible || isCcCompatible; - // #5420: hide model listing for tool-only providers (web search / web fetch), - // not just `-search`-suffixed ids. Declared serviceKinds come from the static - // provider catalog (e.g. firecrawl → ["webFetch"]); compatible providers resolve - // to null here and fall through to the empty-kinds check (model listing stays on). + // #5420: hide model listing for tool-only providers, not just `-search` ids. const declaredServiceKinds = ( resolveDashboardProviderInfo(providerId) as { serviceKinds?: readonly string[] } | null )?.serviceKinds; @@ -224,8 +221,6 @@ export default function ProviderDetailPageClient() { // Prefer synced API-discovered models when available, then merge built-ins // and user-managed custom models without duplicating IDs. const models = useMemo(() => { - // Universal: merge built-in registry models with API-synced models and - // user-managed custom models for ALL providers (was previously Gemini-only). // Synced models keep their full property spread so provider-specific fields // (e.g. Gemini's `supportedGenerationMethods`) survive into the table. const builtInModels = registryModels.map((model) => ({ diff --git a/src/app/api/providers/[id]/models/discovery/codex.ts b/src/app/api/providers/[id]/models/discovery/codex.ts new file mode 100644 index 0000000000..431b822fbc --- /dev/null +++ b/src/app/api/providers/[id]/models/discovery/codex.ts @@ -0,0 +1,388 @@ +import { + getCodexClientVersion, + getCodexDefaultHeaders, +} from "@omniroute/open-sse/config/codexClient.ts"; + +export const CODEX_MODELS_URL = "https://chatgpt.com/backend-api/codex/models"; +export const CODEX_GITHUB_MODELS_URL = + "https://raw.githubusercontent.com/openai/codex/refs/heads/main/codex-rs/models-manager/models.json"; +export const CODEX_GITHUB_CATALOG_CACHE_TTL_MS = 5 * 60 * 1000; + +type JsonRecord = Record; + +export type CodexDiscoveryModel = { + id: string; + name: string; + owned_by: "codex"; + apiFormat: "responses"; + supportedEndpoints: ["responses"]; + inputTokenLimit?: number; + outputTokenLimit?: number; + description?: string; + supportsThinking?: boolean; + supportsVision?: boolean; +}; + +export type CodexModelsFetch = ( + input: string, + init: { + method: "GET"; + headers: Record; + } +) => Promise; + +type CodexGithubCatalogCache = { + models: CodexDiscoveryModel[]; + etag?: string; + expiresAt: number; +}; + +let codexGithubCatalogCache: CodexGithubCatalogCache | null = null; + +function asRecord(value: unknown): JsonRecord { + return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {}; +} + +function toNonEmptyString(value: unknown): string | null { + return typeof value === "string" && value.trim().length > 0 ? value.trim() : null; +} + +function firstPositiveNumber(...candidates: unknown[]): number | undefined { + for (const candidate of candidates) { + if (typeof candidate === "number" && Number.isFinite(candidate) && candidate > 0) { + return candidate; + } + } + return undefined; +} + +function parseVersionParts(version: string): number[] | null { + const parts = version + .trim() + .split(".") + .map((part) => Number(part)); + return parts.length > 0 && parts.every((part) => Number.isInteger(part) && part >= 0) + ? parts + : null; +} + +function compareVersions(left: string, right: string): number { + const leftParts = parseVersionParts(left); + const rightParts = parseVersionParts(right); + if (!leftParts || !rightParts) return 0; + + const length = Math.max(leftParts.length, rightParts.length); + for (let index = 0; index < length; index += 1) { + const a = leftParts[index] || 0; + const b = rightParts[index] || 0; + if (a !== b) return a - b; + } + return 0; +} + +export function buildCodexModelsUrl(clientVersion = getCodexClientVersion()): string { + const url = new URL(CODEX_MODELS_URL); + url.searchParams.set("client_version", clientVersion); + return url.toString(); +} + +function getCodexModelItems(payload: unknown): unknown[] { + const record = asRecord(payload); + if (Array.isArray(record.models)) return record.models; + if (Array.isArray(record.data)) return record.data; + if (Array.isArray(payload)) return payload; + + const objectItems = Object.entries(record) + .filter(([, value]) => value && typeof value === "object" && !Array.isArray(value)) + .map(([key, value]) => ({ id: key, ...asRecord(value) })); + return objectItems.length > 0 ? objectItems : []; +} + +function shouldImportCodexModel(record: JsonRecord): boolean { + if (toNonEmptyString(record.visibility)?.toLowerCase() === "hide") return false; + if (record.supported_in_api === false || record.supportedInApi === false) return false; + + const minimalClientVersion = + toNonEmptyString(record.minimal_client_version) || + toNonEmptyString(record.minimalClientVersion); + if (minimalClientVersion && compareVersions(minimalClientVersion, getCodexClientVersion()) > 0) { + return false; + } + + return true; +} + +function getCodexModelId(record: JsonRecord): string | null { + return ( + toNonEmptyString(record.slug) || toNonEmptyString(record.id) || toNonEmptyString(record.model) + ); +} + +function getCodexModelName(record: JsonRecord, id: string): string { + return ( + toNonEmptyString(record.display_name) || + toNonEmptyString(record.displayName) || + toNonEmptyString(record.name) || + toNonEmptyString(record.title) || + id + ); +} + +function recordSupportsThinking(record: JsonRecord): boolean { + return ( + Array.isArray(record.supported_reasoning_levels) && record.supported_reasoning_levels.length > 0 + ); +} + +function isImageModality(modality: unknown): boolean { + return toNonEmptyString(modality)?.toLowerCase() === "image"; +} + +function recordSupportsVision(record: JsonRecord): boolean { + return Array.isArray(record.input_modalities) && record.input_modalities.some(isImageModality); +} + +function buildCodexDiscoveryModel(record: JsonRecord): CodexDiscoveryModel | null { + if (!shouldImportCodexModel(record)) return null; + + const id = getCodexModelId(record); + if (!id) return null; + + const topProvider = asRecord(record.top_provider); + const limits = asRecord(record.limits); + const model: CodexDiscoveryModel = { + id, + name: getCodexModelName(record, id), + owned_by: "codex", + apiFormat: "responses", + supportedEndpoints: ["responses"], + }; + const inputTokenLimit = firstPositiveNumber( + record.inputTokenLimit, + record.maxInputTokens, + record.max_input_tokens, + record.contextLength, + record.context_length, + record.context_window, + record.max_context_window, + topProvider.context_length, + limits.input_tokens, + limits.inputTokenLimit, + limits.max_input_tokens + ); + const outputTokenLimit = firstPositiveNumber( + record.outputTokenLimit, + record.maxOutputTokens, + record.max_output_tokens, + topProvider.max_completion_tokens, + limits.output_tokens, + limits.outputTokenLimit, + limits.max_output_tokens + ); + const description = toNonEmptyString(record.description); + + if (typeof inputTokenLimit === "number") model.inputTokenLimit = inputTokenLimit; + if (typeof outputTokenLimit === "number") model.outputTokenLimit = outputTokenLimit; + if (description) model.description = description; + if (recordSupportsThinking(record)) model.supportsThinking = true; + if (recordSupportsVision(record)) model.supportsVision = true; + + return model; +} + +export function normalizeCodexModelsResponse(payload: unknown): CodexDiscoveryModel[] { + const deduped = new Map(); + + for (const item of getCodexModelItems(payload)) { + const model = buildCodexDiscoveryModel(asRecord(item)); + if (model) deduped.set(model.id, model); + } + + return Array.from(deduped.values()); +} + +export function normalizeCodexGithubCatalogResponse(payload: unknown): CodexDiscoveryModel[] { + return normalizeCodexModelsResponse(payload); +} + +export function clearCodexGithubCatalogCacheForTests(): void { + codexGithubCatalogCache = null; +} + +function getFreshCodexGithubCatalogCache( + now: number, + cacheTtlMs: number +): CodexDiscoveryModel[] | null { + if (cacheTtlMs > 0 && codexGithubCatalogCache?.expiresAt > now) { + return codexGithubCatalogCache.models; + } + return null; +} + +function buildCodexGithubCatalogHeaders(): Record { + const headers: Record = { + Accept: "application/json", + "Content-Type": "application/json", + }; + if (codexGithubCatalogCache?.etag) { + headers["If-None-Match"] = codexGithubCatalogCache.etag; + } + return headers; +} + +function getNotModifiedCodexGithubCatalog( + response: Response, + now: number, + cacheTtlMs: number +): CodexDiscoveryModel[] | null { + if (response.status !== 304 || !codexGithubCatalogCache) return null; + + codexGithubCatalogCache = { + ...codexGithubCatalogCache, + expiresAt: now + cacheTtlMs, + }; + return codexGithubCatalogCache.models; +} + +function storeCodexGithubCatalogCache( + models: CodexDiscoveryModel[], + response: Response, + now: number, + cacheTtlMs: number +): void { + const etag = toNonEmptyString(response.headers.get("etag")); + codexGithubCatalogCache = { + models, + ...(etag ? { etag } : {}), + expiresAt: now + cacheTtlMs, + }; +} + +type CodexLocalCatalogModel = { + id: string; + name?: string; + apiFormat?: string; + supportedEndpoints?: string[]; + contextLength?: number; + maxInputTokens?: number; + maxOutputTokens?: number; +}; + +function localCatalogModelToCodexDiscoveryModel( + model: CodexLocalCatalogModel +): CodexDiscoveryModel { + const inputTokenLimit = firstPositiveNumber(model.maxInputTokens, model.contextLength); + const outputTokenLimit = firstPositiveNumber(model.maxOutputTokens); + return { + id: model.id, + name: model.name || model.id, + owned_by: "codex", + apiFormat: "responses", + supportedEndpoints: ["responses"], + ...(typeof inputTokenLimit === "number" ? { inputTokenLimit } : {}), + ...(typeof outputTokenLimit === "number" ? { outputTokenLimit } : {}), + }; +} + +export function mergeCodexLiveModelsWithLocalCatalog( + liveModels: CodexDiscoveryModel[], + localCatalogModels: CodexLocalCatalogModel[] +): CodexDiscoveryModel[] { + const merged = new Map(); + + for (const liveModel of liveModels) { + merged.set(liveModel.id, liveModel); + } + + for (const localModel of localCatalogModels) { + if (!localModel.id) continue; + const normalizedLocal = localCatalogModelToCodexDiscoveryModel(localModel); + const existing = merged.get(localModel.id); + merged.set(localModel.id, existing ? { ...normalizedLocal, ...existing } : normalizedLocal); + } + + return Array.from(merged.values()); +} + +export function enrichCodexModelsFromGithubCatalog( + models: CodexDiscoveryModel[], + githubCatalogModels: CodexDiscoveryModel[] +): CodexDiscoveryModel[] { + const byId = new Map(githubCatalogModels.map((model) => [model.id, model])); + return models.map((model) => { + const githubModel = byId.get(model.id); + return githubModel ? { ...githubModel, ...model } : model; + }); +} + +export async function fetchCodexDiscoveryModels({ + accessToken, + providerSpecificData, + fetchImpl, +}: { + accessToken: string | null; + providerSpecificData?: Record | null; + fetchImpl: CodexModelsFetch; +}): Promise { + if (!accessToken) return null; + + try { + const workspaceId = + toNonEmptyString(providerSpecificData?.workspaceId) || + toNonEmptyString(providerSpecificData?.chatgptAccountId) || + toNonEmptyString(providerSpecificData?.accountId); + const headers: Record = { + ...getCodexDefaultHeaders(), + Accept: "application/json", + "Content-Type": "application/json", + Authorization: `Bearer ${accessToken}`, + originator: "codex_cli_rs", + }; + if (workspaceId) headers["chatgpt-account-id"] = workspaceId; + + const response = await fetchImpl(buildCodexModelsUrl(), { + method: "GET", + headers, + }); + + if (!response.ok) return null; + + const models = normalizeCodexModelsResponse(await response.json()); + return models.length > 0 ? models : null; + } catch { + return null; + } +} + +export async function fetchCodexGithubCatalogModels({ + fetchImpl, + now = Date.now(), + cacheTtlMs = CODEX_GITHUB_CATALOG_CACHE_TTL_MS, +}: { + fetchImpl: CodexModelsFetch; + now?: number; + cacheTtlMs?: number; +}): Promise { + const cachedModels = getFreshCodexGithubCatalogCache(now, cacheTtlMs); + if (cachedModels) return cachedModels; + + try { + const response = await fetchImpl(CODEX_GITHUB_MODELS_URL, { + method: "GET", + headers: buildCodexGithubCatalogHeaders(), + }); + + const notModifiedModels = getNotModifiedCodexGithubCatalog(response, now, cacheTtlMs); + if (notModifiedModels) return notModifiedModels; + + if (!response.ok) return null; + + const models = normalizeCodexGithubCatalogResponse(await response.json()); + if (models.length === 0) return null; + + storeCodexGithubCatalogCache(models, response, now, cacheTtlMs); + return models; + } catch { + return codexGithubCatalogCache?.models || null; + } +} diff --git a/src/app/api/providers/[id]/models/route.ts b/src/app/api/providers/[id]/models/route.ts index 9d4c92075e..65a15ed642 100755 --- a/src/app/api/providers/[id]/models/route.ts +++ b/src/app/api/providers/[id]/models/route.ts @@ -97,6 +97,12 @@ import { type ProviderModelsConfigEntry, PROVIDER_MODELS_CONFIG, } from "./discovery/providerModelsConfig"; +import { + enrichCodexModelsFromGithubCatalog, + fetchCodexDiscoveryModels, + fetchCodexGithubCatalogModels, + mergeCodexLiveModelsWithLocalCatalog, +} from "./discovery/codex"; /** * GET /api/providers/[id]/models - Get models list from provider @@ -330,14 +336,16 @@ export async function GET( const buildDiscoveryFallbackResponse = ({ cacheWarning = "API unavailable — using cached catalog", localWarning = "API unavailable — using local catalog", + localIntentional = false, }: { cacheWarning?: string; localWarning?: string; + localIntentional?: boolean; } = {}) => { if (cachedDiscoveryModels.length > 0) { return buildCachedDiscoveryResponse(cacheWarning); } - return buildLocalCatalogResponse(localWarning); + return buildLocalCatalogResponse(localWarning, localIntentional); }; const buildDiscoveryErrorFallbackResponse = ( @@ -1712,6 +1720,71 @@ export async function GET( }); } + if (provider === "codex") { + const cachedResponse = maybeReturnCachedDiscovery(); + if (cachedResponse) return cachedResponse; + + const autoFetchDisabledResponse = maybeReturnAutoFetchDisabled(); + if (autoFetchDisabledResponse) return autoFetchDisabledResponse; + + const liveModels = await fetchCodexDiscoveryModels({ + accessToken: accessToken || null, + providerSpecificData: connection.providerSpecificData, + fetchImpl: (url, init) => + safeOutboundFetch(url, { + ...SAFE_OUTBOUND_FETCH_PRESETS.modelsDiscovery, + guard: getProviderOutboundGuard(), + proxyConfig: proxy, + ...init, + }), + }); + const githubCatalogModels = await fetchCodexGithubCatalogModels({ + fetchImpl: (url, init) => + safeOutboundFetch(url, { + ...SAFE_OUTBOUND_FETCH_PRESETS.modelsDiscovery, + guard: "public-only", + proxyConfig: proxy, + ...init, + }), + }); + const staticCodexCatalog = mergeLocalCatalogModels( + getModelsByProviderId("codex") || [], + getStaticModelsForProvider("codex") || [] + ); + + if (liveModels && liveModels.length > 0) { + const enrichedLiveModels = + githubCatalogModels && githubCatalogModels.length > 0 + ? enrichCodexModelsFromGithubCatalog(liveModels, githubCatalogModels) + : liveModels; + return buildApiDiscoveryResponse( + mergeCodexLiveModelsWithLocalCatalog(enrichedLiveModels, staticCodexCatalog) + ); + } + + if (githubCatalogModels && githubCatalogModels.length > 0) { + return buildApiDiscoveryResponse( + mergeCodexLiveModelsWithLocalCatalog(githubCatalogModels, staticCodexCatalog), + "Codex live catalog unavailable — using GitHub model catalog" + ); + } + + const fallback = buildDiscoveryFallbackResponse({ + cacheWarning: "Codex live catalog unavailable — using cached catalog", + localWarning: "Codex live and GitHub catalogs unavailable — using local catalog", + localIntentional: true, + }); + if (fallback) return fallback; + return buildResponse({ + provider, + connectionId, + models: [], + source: "local_catalog", + intentional: true, + warning: "Codex live and GitHub catalogs unavailable — using local catalog", + }); + } + const localCatalog = mergeLocalCatalogModels(registryCatalogModels, specialtyCatalogModels); if (!config && localCatalog.length > 0) { return buildResponse({ diff --git a/src/i18n/messages/pt-BR.json b/src/i18n/messages/pt-BR.json index 7a067a450b..ad6e79f6de 100644 --- a/src/i18n/messages/pt-BR.json +++ b/src/i18n/messages/pt-BR.json @@ -25,6 +25,7 @@ "noData": "Nenhum dado disponível", "nothingHere": "Nothing here yet", "configure": "Configurar", + "manualConfig": "Configuração manual", "manage": "Gerenciar", "name": "Nome", "actions": "Ações", @@ -33,6 +34,7 @@ "model": "Modelo", "models": "modelos", "provider": "Provedor", + "unknownProvider": "Provedor desconhecido", "account": "Conta", "time": "Tempo", "details": "Detalhes", @@ -843,6 +845,7 @@ "batchActionRetryError": "Falha ao refazer requisições com falha. Tente novamente.", "batchConceptRetentionNote": "Arquivos de resultado e erro ficam retidos por 30 dias (Anthropic: 29 dias)" }, + "disabled": "Desabilitado", "featureFlagOmnirouteEmergencyFallbackDescription": "__MISSING__:Route budget-exhausted requests to the emergency free fallback provider/model.", "featureFlagArenaEloSyncEnabledDescription": "__MISSING__:Enable periodic Arena AI leaderboard ELO sync for model intelligence rankings.", "sidebar": { @@ -1115,7 +1118,10 @@ "dragReorderItem": "__MISSING__:Drag to reorder", "cannotHide": "__MISSING__:This item cannot be hidden", "alwaysVisible": "__MISSING__:Always visible", - "groupSeparatorLabel": "__MISSING__:Separator" + "groupSeparatorLabel": "__MISSING__:Separator", + "discovery": "Descoberta", + "discoverySubtitle": "Buscar acesso gratuito em provedores", + "chaosConfig": "Chaos Mode" }, "webhooks": { "title": "Webhooks", @@ -1761,6 +1767,8 @@ "copyMaskedKey": "Copiar chave mascarada", "keyOnlyAvailableAtCreation": "Full key available only at creation time — copy it when you first create the key", "modelsCount": "{count, plural, one {# modelo} other {# modelos}}", + "devicesCount": "{count, plural, one {# dispositivo} other {# dispositivos}}", + "devicesTooltip": "{count, plural, one {# dispositivo distinto (IP/User-Agent) visto com esta chave (últimos 30 min)} other {# dispositivos distintos (IP/User-Agent) vistos com esta chave (últimos 30 min)}}", "lastUsedOn": "Último: {date}", "viewCostsFor": "__MISSING__:View costs for {name}", "editPermissions": "Editar permissões", @@ -1773,6 +1781,8 @@ "models": "{count} modelos", "permissionsTitle": "Permissões: {name}", "allowAllDesc": "Esta chave pode acessar todos os modelos disponíveis.", + "restrictLoading": "Carregando catálogo de modelos…", + "restrictCatalogUnavailable": "Catálogo de modelos indisponível; esta chave tem {selectedCount} restrições de modelo selecionadas.", "restrictDesc": "Esta chave pode acessar {selectedCount} de {totalModels} modelos.", "selectedCount": "{count} selecionados", "maxActiveSessions": "Máximo de sessões ativas", @@ -1856,14 +1866,16 @@ "webSearch": "Busca Web", "webFetch": "Web Fetch", "video": "Vídeo", - "music": "Música" + "music": "Música", + "ocr": "OCR" }, "noProviders": "Nenhum provedor configurado para este tipo.", "addConnection": "Adicionar Conexão", "backToProviders": "Voltar para Provedores", "connections": "{count} Conexões", "noConnections": "Nenhuma conexão ainda — adicione uma na página do provedor.", - "loading": "Carregando..." + "loading": "Carregando...", + "suggestedModels": "Modelos sugeridos pelo provedor" }, "search": { "searchQuery": "Search Query", @@ -2391,6 +2403,14 @@ "strictRandomDesc": "Baralho embaralhado — usa cada modelo uma vez antes de reembaralhar", "fusion": "__MISSING__:Fusion", "fusionDesc": "__MISSING__:Fans the prompt to every panel model in parallel, then a judge model synthesizes one final answer", + "fusionJudgeModel": "Modelo julgador", + "fusionJudgeModelHelp": "Modelo que sintetiza as respostas do painel em uma resposta final única. Deixe vazio para usar o primeiro modelo do painel.", + "fusionMinPanel": "Painel mínimo", + "fusionMinPanelHelp": "Respostas bem-sucedidas do painel necessárias antes que os retardatários recebam uma janela de tolerância (padrão 2).", + "fusionStragglerGraceMs": "Tolerância para retardatários (ms)", + "fusionStragglerGraceMsHelp": "Quanto tempo esperar por modelos lentos do painel após o quórum ser atingido (padrão 8000).", + "fusionPanelHardTimeoutMs": "Timeout máximo do painel (ms)", + "fusionPanelHardTimeoutMsHelp": "Limite absoluto para que um modelo travado não bloqueie o painel inteiro (padrão 90000).", "models": "Modelos", "autoBalance": "Auto-balancear", "advancedSettings": "Configurações Avançadas", @@ -2500,7 +2520,8 @@ "queueTimeout": "Tempo máximo em fila antes de expirar no round-robin.", "failoverBeforeRetry": "Quando ativado, qualquer erro upstream aciona o failover imediato para o próximo destino combinado, ignorando todas as novas tentativas e URLs substitutos.", "maxSetRetries": "Número de vezes para tentar novamente o destino completo definido quando todos os alvos falharem. 0 = nenhuma nova tentativa de nível definido.", - "setRetryDelayMs": "Atraso entre novas tentativas de nível definido, dando tempo para que problemas transitórios sejam resolvidos." + "setRetryDelayMs": "Atraso entre novas tentativas de nível definido, dando tempo para que problemas transitórios sejam resolvidos.", + "disableSessionStickiness": "Alterna para uma conexão diferente a cada requisição, em vez de fixar toda a conversa em uma conexão pelo hash da primeira mensagem. Sobrescreve o padrão global. Deixe em Herdar para preservar acertos de cache de prompt em conversas com múltiplos turnos." }, "templatesTitle": "Templates rápidos", "templatesDescription": "Aplique um perfil inicial e depois ajuste modelos e configuração.", @@ -2794,6 +2815,13 @@ "browseLegacyCatalog": "Navegue pelo catálogo combinado legado", "agentFeaturesTitle": "Recursos do agente", "agentFeaturesDescription": "Habilite recursos avançados para agentes usando esta combinação", + "responseValidationTitle": "Validação de resposta", + "responseValidationHelp": "Faz failover para o próximo alvo quando um corpo 200 OK falha nestas verificações (conteúdo do assistente).", + "responseValidationForbidden": "Substrings proibidas (uma por linha)", + "responseValidationRequired": "Substrings obrigatórias (uma por linha)", + "responseValidationMinLength": "Tamanho mínimo do conteúdo (caracteres)", + "responseValidationJsonPaths": "Verificações de JSON-path", + "responseValidationAddCheck": "+ Adicionar verificação", "agentFeaturesSystemMessageOverride": "Substituir mensagem do sistema", "agentFeaturesSystemMessagePlaceholder": "Você é um assistente especialista...", "agentFeaturesSystemMessageHint": "Substituição de mensagens do sistema para agentes", @@ -2807,7 +2835,10 @@ "agentFeaturesContextLengthErrorInteger": "O comprimento do contexto precisa ser um número inteiro válido", "agentFeaturesContextLengthErrorRange": "O comprimento do contexto precisa estar entre 1.000 e 2.000.000", "compressionOverride": "Substituição de compactação", - "modePack": "Pacote de modos" + "modePack": "Pacote de modos", + "disableSessionStickiness": "Desabilitar fixação de sessão", + "sessionStickinessEnabled": "Fixação ativada", + "sessionStickinessDisabled": "Fixação desativada" }, "costs": { "title": "Custos", @@ -3410,6 +3441,10 @@ "description": "Integração opcional com Qdrant para busca semântica escalável", "enableLabel": "Habilitar Qdrant", "enableDesc": "Quando habilitado, Qdrant é usado como vector store principal", + "banner": "Armazenamento vetorial Tier 2 — uma alternativa externa e escalável ao sqlite-vec embutido (Tier 1). Habilite apenas se você tiver um conjunto de memória muito grande ou quiser memória compartilhada entre instâncias; a maioria dos usuários fica bem com o sqlite-vec. Quando habilitado, ele se torna o armazenamento primário e volta automaticamente para o sqlite-vec se ficar inacessível.", + "hostHelp": "Docker local: http://localhost:6333 · Qdrant Cloud: a URL do seu cluster", + "collectionHelp": "Qualquer nome — o OmniRoute o cria no primeiro uso", + "embeddingModelHelp": "Define a dimensão do vetor automaticamente no primeiro uso. Memórias existentes não são retroalimentadas, e trocar o modelo depois que já existem dados exige uma coleção nova.", "testConnection": "Testar conexão", "testing": "Testando...", "statusActive": "Ativo", @@ -3486,6 +3521,7 @@ "noExecutions": "Nenhuma execução encontrada", "enabled": "Ativado", "disabled": "Desativado", + "delete": "Excluir", "version": "Versão", "tableDescription": "Descrição", "skill": "Skill", @@ -4065,6 +4101,8 @@ "prefixHint": "Obrigatório. Prefixo único para nomes de modelos.", "nameHint": "Obrigatório. Um rótulo amigável para este nó.", "baseUrlHint": "Obrigatório. URL base da API do provedor.", + "iconUrlLabel": "URL do ícone", + "iconUrlHint": "Opcional. URL da imagem exibida como ícone deste provedor.", "anthropicPrefixPlaceholder": "ac-prod", "openaiPrefixPlaceholder": "oc-prod", "anthropicBaseUrlPlaceholder": "https://api.anthropic.com/v1", @@ -4176,6 +4214,8 @@ "optional": "Opcional", "anthropicCompatibleName": "Compatível Anthropic", "openaiCompatibleName": "Compatível OpenAI", + "compatibleDefaultModelLabel": "Modelo padrão", + "compatibleDefaultModelHint": "Digite o ID do modelo exatamente como seu endpoint compatível espera. Este modelo será salvo como o padrão da conexão.", "failedImportModels": "Falha ao importar modelos", "noModelsReturnedFromEndpoint": "Nenhum modelo retornado do endpoint /models.", "importingModelsProgress": "Importando {current} de {total} modelos...", @@ -4282,10 +4322,29 @@ "targetFormatAuto": "__MISSING__:Default (auto)", "targetFormatGemini": "__MISSING__:Gemini", "targetFormatAntigravity": "__MISSING__:Antigravity", - "contextWindowOverrideLabel": "__MISSING__:Context Window Override", - "contextWindowOverridePlaceholder": "__MISSING__:e.g. 131072", - "contextWindowOverrideHint": "__MISSING__:Manually set this model's real context window (tokens) when the provider misreports it. Wins over auto-detected/catalog values and prevents combo routing from dropping the model.", - "contextWindowOverrideInvalid": "__MISSING__:Context window override must be a positive whole number of tokens", + "compatParamFiltersLabel": "Filtros de parâmetros", + "compatBlockedParamsHint": "Parâmetros bloqueados (removidos das requisições)", + "compatAllowedParamsHint": "Parâmetros permitidos (readicionados após o bloqueio)", + "paramFiltersSectionTitle": "Filtros de parâmetros", + "paramFiltersSectionHint": "Remove ou readiciona parâmetros da requisição antes de enviá-la a este provedor. Usado para evitar erros 400 de provedores que rejeitam certos parâmetros (ex.: NVIDIA NIM rejeita thinking).", + "paramFiltersBlockedLabel": "Parâmetros bloqueados", + "paramFiltersBlockedHint": "Estes parâmetros são removidos das requisições de saída (lista de bloqueio).", + "paramFiltersAllowedLabel": "Parâmetros permitidos", + "paramFiltersAllowedHint": "Estes parâmetros são readicionados após a remoção pela lista de bloqueio (somente se o cliente os enviou).", + "paramFiltersAutoLearnLabel": "Aprender automaticamente com erros 400", + "paramFiltersAutoLearnHint": "Quando habilitado, se o upstream retornar 400 com \"Unsupported parameter: X\", o parâmetro é automaticamente adicionado à lista de bloqueio e a requisição é refeita.", + "paramFiltersSaving": "Salvando…", + "paramFiltersSaveChanges": "Salvar alterações", + "paramFiltersResetToDefault": "Restaurar padrão", + "paramFiltersLoadError": "Falha ao carregar configuração de filtro de parâmetros: {error}", + "paramFiltersSaveSuccess": "Configuração de filtro de parâmetros salva", + "paramFiltersSaveError": "Falha ao salvar configuração de filtro de parâmetros: {error}", + "paramFiltersResetSuccess": "Configuração de filtro de parâmetros restaurada ao padrão", + "paramFiltersResetError": "Falha ao restaurar configuração de filtro de parâmetros: {error}", + "contextWindowOverrideLabel": "Substituição de Janela de Contexto", + "contextWindowOverridePlaceholder": "ex.: 131072", + "contextWindowOverrideHint": "Define manualmente a janela de contexto real (tokens) deste modelo quando o provedor a reporta incorretamente. Tem prioridade sobre os valores detectados automaticamente/do catálogo e evita que o roteamento de combo descarte o modelo.", + "contextWindowOverrideInvalid": "A substituição de janela de contexto deve ser um número inteiro positivo de tokens", "compatUpstreamHeadersLabel": "Extra upstream headers", "compatUpstreamHeadersHint": "High-privilege setting — same trust level as editing provider API credentials; only trusted admins should use it. Merged after OmniRoute adds auth from the provider API key. If a custom header uses the same name as an existing one (e.g. Authorization), your value fully replaces the auto-generated header (including the Bearer token) — the upstream only sees what you typed, not the key from settings. Misconfiguration can cause 401 or broken upstream auth. One row per header (e.g. extra Authentication for some gateways). Hover or focus the value to preview. Saves on blur, outside click, or closing this panel.", "compatUpstreamHeaderName": "Header name", @@ -4482,6 +4541,8 @@ "modelsPathLabel": "Models Endpoint Path", "modelsPathPlaceholder": "/models", "modelsPathHint": "Custom models path for validation (e.g. /v4/models)", + "clientIdentityLabel": "Identidade do cliente", + "clientIdentityHint": "Opcional. Adiciona cabeçalhos de fingerprint do cliente (ex.: User-Agent) correspondentes a um CLI conhecido, para gateways compatíveis que esperam um.", "statusDeactivated": "Desativado (Manual)", "statusBanned": "Banido / Sandbox Violation", "statusCreditsExhausted": "Saldo Insuficiente", @@ -4604,6 +4665,11 @@ "localProviderBaseUrlHint": "Obrigatório: cole a URL base OpenAI-compatible /v1 do seu {provider} (padrão: {baseUrl}). URLs locais/privadas exigem OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS=true para validação no dashboard.", "localProviders": "Local / Self-Hosted", "maxConcurrentWholeNumberError": "Max concurrent deve ser um número inteiro maior ou igual a 0.", + "m365TierLabel": "Nível do Copilot", + "m365TierHint": "Escolha qual superfície do Microsoft 365 Copilot esta conexão usa. Individual é o BizChat padrão para consumidores; Educação e Empresarial (trabalho) usam as superfícies do respectivo tenant.", + "m365TierIndividualOption": "Individual (padrão)", + "m365TierEduOption": "Educação", + "m365TierEnterpriseOption": "Empresarial / Trabalho", "museSparkWebCookieHint": "Cole o cookie abra_sess do meta.ai. Um cabeçalho de cookie completo também funciona.", "museSparkWebCookiePlaceholder": "Cole o valor de abra_sess", "oauth": "Oauth", @@ -4787,6 +4853,7 @@ "bedrockModelsDescription": "Os modelos Amazon Bedrock têm escopo definido por região da AWS. Importe de /models ou adicione IDs de modelo Bedrock habilitados na região selecionada.", "bedrockModelPlaceholder": "antrópico.claude-sonnet-4-6", "addProviderSessionCookieTitle": "Adicionar cookie de sessão {provider}", + "openWebProviderSite": "Abrir {host}", "addProviderWebTokenTitle": "Adicionar token da web {provider}", "addProviderConnectionTitle": "Adicionar conexão {provider}", "webTokenCredentialLabel": "Token de sessão da Web", @@ -4820,7 +4887,11 @@ "kimiWebLabel": "__MISSING__:Kimi Web", "kimiWebDesc": "__MISSING__:Moonshot AI consumer chat via www.kimi.com (international, Connect-RPC API)", "doubaoWebLabel": "__MISSING__:Doubao Web", - "doubaoWebDesc": "__MISSING__:ByteDance AI chat via doubao.com" + "doubaoWebDesc": "__MISSING__:ByteDance AI chat via doubao.com", + "overrideBaseUrlAdvanced": "Avançado: sobrescrever URL base", + "overrideBaseUrlHint": "Avançado: aponta este provedor embutido para um endpoint personalizado. Deixe em branco para usar o padrão.", + "bulkAddFormatHintCloudflare": "Uma chave por linha. Formato: nome|accountId|apiKey (ID de conta Cloudflare + token de API).", + "lmarenaWebCookieHint": "Abra arena.ai, faça login e depois copie o cabeçalho Cookie completo de uma requisição de rede. Inclua arena-auth-prod-v1.0 e arena-auth-prod-v1.1 (e outros fragmentos, se houver), preferencialmente com cf_clearance. Não cole apenas o cookie vazio arena-auth-prod-v1. Opcional: providerSpecificData.recaptchaV3Token se create-evaluation ainda retornar 403." }, "settings": { "title": "Configurações", @@ -4830,6 +4901,10 @@ "routing": "Roteamento", "cache": "Cache", "resilience": "Resiliência", + "description": "Descrição", + "enable": "Habilitar", + "disable": "Desabilitar", + "update": "Atualizar", "routingSettingsIntro": "Controla como suas solicitações são roteadas, transformadas e enviadas aos provedores de IA.", "routingOpDropParagraphContainsLabel": "Remover parágrafo (contém)", "routingOpDropParagraphStartsWithLabel": "Remover parágrafo (começa com)", @@ -4965,6 +5040,8 @@ "modelsDevInfoOrder": "User Override → models.dev → LiteLLM → Hardcoded Default", "systemTheme": "Tema do Sistema", "debugToggle": "Enable Debug Mode", + "logToolSourcesToggle": "Registrar fontes de ferramentas", + "logToolSourcesDescription": "Emite uma linha de log de diagnóstico por requisição resumindo a contagem de ferramentas e a origem MCP/hospedada/cliente.", "homePinProviderQuotaToHome": "Fixar informações na página inicial", "homeProviderQuotaLimits": "Limites de cota dos provedores", "homeProviderQuotaLimitsDesc": "Fixe o painel de status de cotas dos provedores (com botão Atualizar tudo) no topo da página inicial.", @@ -5358,6 +5435,9 @@ "providerMaxRetriesAria": "{provider} tentativas máximas", "providerTimeoutAria": "timeout de {provider} em ms", "removeProviderOverrideAria": "Remover sobrescrita de {provider}", + "selectProviderPlaceholder": "Selecionar provedor...", + "searchProviderPlaceholder": "Buscar provedores...", + "searchProviderAria": "Buscar provedores", "newProviderNamePlaceholder": "ex.: google, openai...", "newProviderNameAria": "Nome do novo provedor", "retries": "tentativas", @@ -5366,6 +5446,7 @@ "maxNestingDepth": "Profundidade Máx. de Aninhamento", "concurrencyPerModel": "Concorrência / Modelo", "queueTimeout": "Timeout da Fila (ms)", + "queueDepth": "Profundidade da fila", "contextRelayHandoffThreshold": "Limite de Handoff", "contextRelayMaxMessages": "Máx. de Mensagens no Resumo", "contextRelaySummaryModel": "Modelo do Resumo", @@ -5519,6 +5600,22 @@ "maintenance": "Maintenance", "purgeExpiredLogs": "Purgar Logs Expirados", "purgeLogsFailed": "Falha ao purgar logs", + "logsDeleted": "{count, plural, =0 {Nenhum log expirado removido} one {# log expirado removido} other {# logs expirados removidos}}", + "resetUsageData": "Redefinir dados de uso", + "resetUsageDataDesc": "Selecione quanto tempo no passado você deseja excluir dados de uso. Esta ação não pode ser desfeita.", + "resetUsagePeriod_5m": "5 minutos", + "resetUsagePeriod_1h": "1 hora", + "resetUsagePeriod_3h": "3 horas", + "resetUsagePeriod_6h": "6 horas", + "resetUsagePeriod_12h": "12 horas", + "resetUsagePeriod_1d": "1 dia", + "resetUsagePeriod_7d": "7 dias", + "resetUsagePeriod_30d": "30 dias", + "resetUsagePeriod_all": "Todo o período", + "resetUsageSuccess": "{count, plural, =0 {Nenhuma linha de dados de uso excluída} one {Dados de uso redefinidos (# linha excluída)} other {Dados de uso redefinidos (# linhas excluídas)}}", + "resetUsageFailed": "Falha ao redefinir dados de uso", + "reset": "Redefinir", + "resetting": "Redefinindo...", "contextOpt": "Context Optimized", "contextOptDesc": "Routes based on context window requirements and conversation length", "priorityDesc": "Sequential fallback - tries provider 1 first, then provider 2, and so on", @@ -5619,8 +5716,12 @@ "compressionAutoTrigger": "Auto-Trigger Threshold", "compressionCacheTTL": "Cache TTL", "compressionPreserveSystem": "Preserve System Prompt", + "compressionPreserveSystemAlways": "Sempre", + "compressionPreserveSystemWhenNoCache": "Quando não há cache", + "compressionPreserveSystemNever": "Nunca", "compressionCavemanConfig": "Caveman Engine Configuration", "compressionCavemanConfigDesc": "Fine-tune the rule-based compression engine", + "compressionCavemanPanelHint": "Ativado/desativado e o nível são definidos no painel:", "compressionRoles": "Compress Message Roles", "compressionRoleUser": "User", "compressionRoleAssistant": "Assistant", @@ -5827,6 +5928,17 @@ "storageUsageTokenBufferDesc": "__MISSING__:Extra tokens added to reported usage to account for system prompt overhead.", "storageUsageTokenBufferHint": "__MISSING__:Set to 0 to report raw provider token counts. Default: 2000.", "storageUsageTokenBufferCurrent": "__MISSING__:Current: {value}", + "redisLauncherTitle": "Redis local", + "redisLauncherDesc": "Inicie com um clique um contêiner Redis 7 (Podman ou Docker) para cache de resposta, controle de cota e limitação de taxa.", + "redisLauncherRefresh": "Atualizar", + "redisLauncherStop": "Parar", + "redisLauncherLaunching": "Iniciando...", + "redisLauncherLaunch": "Iniciar Redis", + "redisLauncherContainer": "Contêiner", + "redisLauncherRunning": "Em execução", + "redisLauncherReachable": "Acessível", + "redisLauncherError": "Erro: {message}", + "redisLauncherHint": "Equivalente a executar `omniroute redis up`. O contêiner é chamado `omniroute-redis` e escuta em 127.0.0.1:6379.", "compressionSettingsAutoTriggerMode": "Modo de disparo automático", "compressionSettingsMcpDescriptionCompression": "Compressão de descrição MCP", "mcpAccessibilityTitle": "__MISSING__:MCP accessibility output", @@ -5910,6 +6022,12 @@ "claudeFastModeModelCheckbox": "Ativar modo rápido para {model}", "claudeFastModeSaveError": "Falha ao atualizar a configuração do modo rápido do Claude", "authz": { + "cors": { + "wildcard": { + "title": "CORS está aberto para qualquer origem (CORS_ALLOW_ALL=true)", + "desc": "Qualquer site pode chamar a API deste servidor a partir do navegador de um visitante. Use apenas em redes confiáveis — defina origens explícitas em ALLOWED_ORIGINS e desabilite o CORS_ALLOW_ALL em produção." + } + }, "title": "Inventário de Authz", "description": "Classificação de rotas em 5 níveis com política de bypass em tempo real. A leitura mostra a taxonomia completa; as mutações solicitam a senha de gerenciamento.", "loading": "Carregando inventário…", @@ -5998,6 +6116,14 @@ "resilienceEnableServerWaitDesc": "Quando ativado, o OmniRoute aguarda o primeiro resfriamento expirar e tenta novamente automaticamente.", "resilienceMaxAttempts": "Máximo de tentativas", "resilienceMaxWaitPerAttempt": "Espera máxima por tentativa", + "resilienceComboCooldownWaitTitle": "Espera de cooldown do combo de cota compartilhada", + "resilienceComboCooldownWaitDesc": "Somente para combos de cota compartilhada: aguarda um cooldown transitório curto e reenvia em vez de retornar um 429 imediatamente. Nunca espera em quota_exhausted.", + "resilienceComboCooldownWaitToggleDesc": "Somente combos de cota compartilhada; nunca espera em quota_exhausted.", + "resilienceComboCooldownMaxWaitMs": "Espera máxima por tentativa", + "resilienceComboCooldownBudgetMs": "Orçamento total de espera", + "resilienceQuotaShareConcurrencyTitle": "Concorrência por conexão de cota compartilhada", + "resilienceQuotaShareConcurrencyDesc": "Somente para combos de cota compartilhada: quando uma conexão define um limite de Concorrência Máxima, serializa as requisições concorrentes para essa conta de assinatura para que nunca seja inundada além do teto. Requisições excedentes esperam na fila em vez de receber um 429. O limite vem do campo Concorrência Máxima de cada conexão; este botão apenas habilita ou desabilita seu respeito.", + "resilienceQuotaShareConcurrencyToggleDesc": "Somente combos de cota compartilhada; respeita o limite de Concorrência Máxima de cada conexão.", "resilienceProviderCooldownTitle": "__MISSING__:Provider Cooldown", "resilienceProviderCooldownScope": "__MISSING__:All combo requests", "resilienceProviderCooldownTrigger": "__MISSING__:When a provider/connection fails", @@ -6066,7 +6192,39 @@ "modelLockoutExponentialBackoff": "__MISSING__:Exponential Backoff", "modelLockoutExponentialBackoffDescription": "__MISSING__:When enabled, each consecutive failure increases the cooldown duration exponentially.", "modelLockoutMaxBackoffSteps": "__MISSING__:Max Backoff Steps", - "modelLockoutMaxBackoffStepsDescription": "__MISSING__:Maximum number of backoff steps before the cooldown stops growing. The Max Cooldown cap is reached first in most configurations, making this a safety ceiling for when Max Cooldown is raised." + "modelLockoutMaxBackoffStepsDescription": "__MISSING__:Maximum number of backoff steps before the cooldown stops growing. The Max Cooldown cap is reached first in most configurations, making this a safety ceiling for when Max Cooldown is raised.", + "disableSessionStickiness": "Desabilitar fixação de sessão", + "disableSessionStickinessDesc": "Combos round-robin e aleatórios alternam para uma conexão diferente a cada requisição, em vez de fixar toda a conversa em uma conexão pelo hash da primeira mensagem. Deixe desativado para preservar acertos de cache de prompt em conversas com múltiplos turnos. Sobrescritas por combo têm prioridade.", + "accountRoundRobin": "Round Robin", + "accountRoundRobinDesc": "Alterna entre contas para distribuir a carga", + "addKeyValue": "Adicionar valor da chave", + "comboRoundRobin": "Round Robin de Combo", + "comboRoundRobinDesc": "Alterna entre os alvos do combo em vez de sempre começar pelo primeiro", + "comboStickyLimit": "Limite de Fixação do Combo", + "comboStickyLimitDesc": "Chamadas por alvo do combo antes de trocar", + "configured": "configurado", + "modelOverrideLoadFailed": "Falha ao carregar substituições de modelo", + "modelOverrideRemoveFailed": "Falha ao remover substituição de modelo", + "modelOverrideRemoved": "Substituição de modelo removida", + "modelOverrideSaveFailed": "Falha ao salvar substituição de modelo", + "modelOverrideSaved": "Substituição de modelo salva", + "modelOverrideValuePlaceholder": "Valor numérico", + "modelOverridesDesc": "Substitui as capacidades de provedor/modelo usadas pelo roteamento e pela formatação de requisições. Os alvos usam o mesmo formato provedor/modelo dos combos.", + "modelOverridesTitle": "Substituições de Modelo", + "noModelOverrides": "Nenhuma substituição configurada para este modelo.", + "none": "Nenhum", + "providerAccountRoutingDesc": "Substitui a estratégia global de contas para este provedor (paridade com 9router).", + "providerAccountRoutingTitle": "Roteamento multi-conta", + "providerRoutingInheritGlobal": "Herdar padrão global", + "providerRoutingStrategy": "Estratégia de conta", + "routingStrategyAccountSummary": "Distribuindo requisições entre contas com {limit} chamadas por conta.", + "routingStrategyComboFallbackSummary": " Combos usam a estratégia configurada de cada combo (prioridade/fallback padrão).", + "routingStrategyComboSummary": " Combos giram após {limit} chamada(s) por alvo.", + "routingStrategyFillFirstSummary": "Usando contas em ordem de prioridade (Fill First).", + "routingStrategySubtitle": "Compatível com 9router: round-robin de contas, limites de fixação e rotação de combo", + "routingStrategyTitle": "Estratégia de Roteamento", + "searchModelOverrideTargets": "Buscar provedor/modelo...", + "selectedModel": "Modelo selecionado" }, "contextRtk": { "title": "Motor RTK", @@ -6639,6 +6797,8 @@ "autoRefresh": "Atualização automática", "refreshAll": "Atualizar tudo", "loadingQuotas": "Carregando...", + "showMoreQuotas": "Mostrar mais {count}", + "showLessQuotas": "Mostrar menos", "account": "Conta", "modelQuotas": "Cotas de Modelo", "lastUsed": "Last Refreshed", @@ -6689,6 +6849,11 @@ "resetsIn": "Reseta em", "editCutoffs": "Editar pontos de corte", "forceRefresh": "Atualizar agora", + "resetCreditsLabel": "Créditos de redefinição", + "redeemResetCredit": "Resgatar redefinição", + "confirmRedeemResetCredit": "Resgatar um crédito de redefinição do Codex para esta conta? Isso consome um crédito de redefinição.", + "resetCreditRedeemed": "Redefinição resgatada", + "resetCreditRedeemFailed": "Falha ao resgatar crédito de redefinição", "suiteBuilderSaveFailed": "Falha ao salvar a suíte customizada", "clone": "Clonar", "exportSuite": "Exportar", @@ -7841,7 +8006,37 @@ "bulkImportMaxExceeded": "Máximo de 100 proxies por importação", "bulkImportPreview": "Visualização", "clearAssignment": "(atribuição clara)", - "bulkProxyAssignment": "Atribuição de proxy em massa" + "bulkProxyAssignment": "Atribuição de proxy em massa", + "testAll": "Testar todos", + "errorTestFailed": "Falha ao testar proxies", + "batchSelectedCount": "{count} selecionado(s)", + "batchDeleteSelected": "Excluir {count} selecionado(s)", + "batchActivateSelected": "Habilitar {count} selecionado(s)", + "testPassed": "✓ OK", + "close": "Fechar", + "managePool": "Gerenciar pool", + "poolTitle": "Pool e rotação de proxies", + "poolDescription": "Anexe múltiplos proxies a um escopo e alterne os IPs de saída entre eles. Um escopo com um único proxy se comporta exatamente como uma atribuição simples.", + "poolScopeIdLabel": "ID do escopo", + "poolScopeIdPlaceholder": "id do provedor / id da conexão / id do combo", + "poolScopeIdRequired": "Um ID de escopo é obrigatório para escopo de provedor, conta ou combo.", + "poolLoad": "Carregar pool", + "poolLoadFailed": "Falha ao carregar o pool de proxies", + "poolStrategyLabel": "Estratégia de rotação", + "poolStrategyHint": "round-robin percorre os membros em ordem; aleatório escolhe uniformemente; fixo mantém um membro por uma janela antes de avançar.", + "poolStrategyFailed": "Falha ao atualizar a estratégia de rotação", + "strategyRoundRobin": "Round-robin", + "strategyRandom": "Aleatório", + "strategySticky": "Fixo", + "poolMembersLabel": "Membros do pool ({count})", + "poolNoMembers": "Ainda não há proxies neste pool.", + "poolRemove": "Remover", + "poolRemoveFailed": "Falha ao remover o proxy do pool", + "poolAddLabel": "Adicionar um proxy", + "poolAddMember": "Adicionar", + "poolAddFailed": "Falha ao adicionar o proxy ao pool", + "poolSelectProxy": "Selecionar um proxy…", + "strategyLatency": "Otimizado por latência" }, "playground": { "title": "Title", @@ -7903,6 +8098,10 @@ "topP": "Top-p", "presencePenalty": "Penalidade de presença", "frequencyPenalty": "Penalidade de frequência", + "reasoningLabel": "Raciocínio", + "thinking": "Pensando", + "effort": "Esforço", + "effortDefault": "Padrão", "seedPlaceholder": "Aleatório (deixar vazio)", "presetsLabel": "Presets", "loadPreset": "Carregar preset", @@ -8380,6 +8579,7 @@ "saveMarketplaceUrl": "__MISSING__:Save & Reload", "marketplaceUrlSaved": "__MISSING__:Marketplace URL updated", "marketplaceEmpty": "__MISSING__:No plugins found in marketplace.", + "marketplaceInstallComingSoon": "Instalações via marketplace em breve.", "verified": "__MISSING__:Verified", "install": "__MISSING__:Install", "installedFromMarketplace": "__MISSING__:Plugin {name} installed!", @@ -8512,6 +8712,7 @@ "restartServer": "Reiniciar", "trustCert": "Confiar Cert", "downloadCert": "Baixar Cert", + "certManualTitle": "O certificado não pôde ser instalado automaticamente (ex.: dentro de um contêiner). A ponte ainda pode ser executada — confie na CA manualmente:", "regenerateCert": "Regenerar Cert", "starting": "Iniciando…", "stopping": "Parando…", @@ -8938,6 +9139,68 @@ "colScore": "__MISSING__:Score", "colAvgScore": "__MISSING__:Avg Score", "colModels": "__MISSING__:Models", - "colType": "__MISSING__:Type" + "colType": "__MISSING__:Type", + "filterConfiguredOnly": "Somente configurados", + "filterAvailableOnly": "Somente disponíveis", + "filterAvailableOnlyHelp": "Ocultar provedores cujas conexões estão todas limitadas por taxa ou sem cota.", + "configuredOnly": "Somente configurados", + "configuredOnlyHint": "Mostrar apenas provedores com conexões ativas", + "noConfiguredProviders": "Nenhum provedor configurado encontrado. Adicione uma conexão de provedor primeiro.", + "colConfigured": "Status" + }, + "discovery": { + "title": "Descoberta de provedores", + "subtitle": "Busque métodos de acesso gratuito/ilimitado em provedores e revise os achados. Opcional, somente local.", + "scanLabel": "Provedor para buscar", + "scanPlaceholder": "ex.: huggingchat", + "scan": "Buscar", + "scanning": "Buscando…", + "scanQueued": "Busca concluída para {provider}.", + "scanFailed": "Falha na busca.", + "loadFailed": "Falha ao carregar os resultados da busca.", + "localOnlyNote": "Esta ferramenta é somente local (loopback). As buscas são executadas nesta máquina e nunca são acessíveis remotamente.", + "verify": "Verificar", + "verifyFailed": "Falha ao verificar o achado.", + "delete": "Excluir", + "deleteFailed": "Falha ao excluir o achado.", + "deleteTitle": "Excluir resultado de busca", + "deleteConfirm": "Excluir o achado de busca para {provider}? Isso não pode ser desfeito.", + "emptyTitle": "Ainda não há resultados de busca", + "emptyDescription": "Execute uma busca acima para procurar métodos de acesso gratuito em um provedor.", + "risk": "Risco", + "method": "Método", + "auth": "Autenticação", + "feasibility": "Viabilidade", + "models": "Modelos" + }, + "chaosConfig": { + "addProvider": "Adicionar Provedor", + "configError": "Falha ao salvar a configuração do Chaos Mode", + "configReset": "Restaurar Padrões", + "configSaved": "Configuração do Chaos Mode salva com sucesso", + "enableChaos": "Ativar Chaos Mode", + "enableChaosDesc": "Permitir que chaves de API com o Chaos Mode ativado usem este recurso", + "keyPermission": "Acesso ao Chaos Mode", + "keyPermissionDesc": "Permitir que esta chave de API use o Chaos Mode (execução paralela multi-modelo)", + "loadingProviderModels": "Carregando provedores...", + "mode": "Modo Padrão", + "modeCollaborative": "Colaborativo", + "modeCollaborativeDesc": "Os modelos encadeiam saídas — cada um vê o resultado anterior", + "modeParallel": "Paralelo", + "modeParallelDesc": "Todos os modelos rodam simultaneamente — resultados mais rápidos", + "modelId": "Modelo", + "pageSubtitle": "Execute múltiplos modelos de IA em paralelo ou de forma colaborativa na mesma tarefa", + "pageTitle": "Chaos Mode", + "providerId": "Provedor", + "providerOverrides": "Substituições de Provedor", + "providerOverridesDesc": "Selecione modelos específicos por provedor para o Chaos Mode", + "removeProvider": "Remover", + "saveConfig": "Salvar Configuração", + "systemPrompt": "Prompt de Sistema (opcional)", + "systemPromptDesc": "Instruções personalizadas para todas as instâncias de modelo do Chaos Mode", + "testButton": "Testar Chaos Mode", + "testTask": "Escreva um poema curto sobre inteligência artificial", + "timeout": "Tempo Limite (ms)", + "timeoutDesc": "Tempo máximo por chamada de modelo (5000-600000ms)" } } diff --git a/src/lib/db/interceptionRules.ts b/src/lib/db/interceptionRules.ts new file mode 100644 index 0000000000..faff5a1fc3 --- /dev/null +++ b/src/lib/db/interceptionRules.ts @@ -0,0 +1,203 @@ +/** + * db/interceptionRules.ts — Per-model web-search / web-fetch interception rules (#3384). + * + * CRUD against the key_value table under namespace "interception_rules". Follows the + * established key_value pattern from paramFilters.ts / databaseSettings.ts. + * + * Resolution precedence (see resolveInterceptSearch): per-model rule > provider-level + * rule > undefined (caller falls back to the existing native-bypass defaults). + */ + +import { getDbInstance } from "./core"; + +const NAMESPACE = "interception_rules"; + +// ── Types ─────────────────────────────────────────────────────────────────── + +export type FetchInterceptionBackend = "firecrawl" | "jina" | "tavily"; + +export interface ModelInterceptionRule { + /** true = route through OmniRoute's /v1/search; false = force native passthrough. */ + interceptSearch?: boolean; + /** true = route through OmniRoute's /v1/web/fetch; false = force native passthrough. */ + interceptFetch?: boolean; + fetchBackend?: FetchInterceptionBackend; + fetchProxyUrl?: string; +} + +export interface ProviderInterceptionRules { + /** Provider-level default, used when a model has no override. */ + interceptSearch?: boolean; + interceptFetch?: boolean; + fetchBackend?: FetchInterceptionBackend; + fetchProxyUrl?: string; + /** Per-model overrides (stricter/looser than provider-level). */ + models?: Record; +} + +// ── Cache ─────────────────────────────────────────────────────────────────── + +let rulesCache: Map | null = null; + +function invalidateCache(): void { + rulesCache = null; +} + +// ── Helpers ───────────────────────────────────────────────────────────────── + +function isRecord(value: unknown): value is Record { + return value !== null && typeof value === "object" && !Array.isArray(value); +} + +function toNormalizedString(value: unknown): string | null { + return typeof value === "string" && value.trim().length > 0 ? value.trim() : null; +} + +function toOptionalBool(value: unknown): boolean | undefined { + return typeof value === "boolean" ? value : undefined; +} + +function toFetchBackend(value: unknown): FetchInterceptionBackend | undefined { + return value === "firecrawl" || value === "jina" || value === "tavily" ? value : undefined; +} + +function parseStoredValue(raw: unknown): unknown { + if (typeof raw !== "string") return raw; + try { + return JSON.parse(raw); + } catch { + return raw; + } +} + +function toModelInterceptionRule(raw: unknown): ModelInterceptionRule | null { + if (!isRecord(raw)) return null; + const rule: ModelInterceptionRule = { + interceptSearch: toOptionalBool(raw.interceptSearch), + interceptFetch: toOptionalBool(raw.interceptFetch), + fetchBackend: toFetchBackend(raw.fetchBackend), + fetchProxyUrl: toNormalizedString(raw.fetchProxyUrl) ?? undefined, + }; + const hasAnyField = Object.values(rule).some((v) => v !== undefined); + return hasAnyField ? rule : null; +} + +function toModelInterceptionRules(raw: unknown): Record { + const models: Record = {}; + if (!isRecord(raw)) return models; + for (const [modelId, val] of Object.entries(raw)) { + const rule = toModelInterceptionRule(val); + if (rule) models[modelId] = rule; + } + return models; +} + +function toProviderInterceptionRules(raw: unknown): ProviderInterceptionRules | null { + if (!isRecord(raw)) return null; + const models = toModelInterceptionRules(raw.models); + return { + interceptSearch: toOptionalBool(raw.interceptSearch), + interceptFetch: toOptionalBool(raw.interceptFetch), + fetchBackend: toFetchBackend(raw.fetchBackend), + fetchProxyUrl: toNormalizedString(raw.fetchProxyUrl) ?? undefined, + models: Object.keys(models).length > 0 ? models : undefined, + }; +} + +// ── Read ──────────────────────────────────────────────────────────────────── + +function readNamespace(namespace: string): Record { + const db = getDbInstance(); + const rows = db + .prepare("SELECT key, value FROM key_value WHERE namespace = ?") + .all(namespace) as Array<{ key: string; value: string }>; + + const values: Record = {}; + for (const row of rows) { + values[row.key] = parseStoredValue(row.value); + } + return values; +} + +function loadAllRules(): Map { + const raw = readNamespace(NAMESPACE); + const map = new Map(); + for (const [key, value] of Object.entries(raw)) { + const parsed = toProviderInterceptionRules(value); + if (parsed) map.set(key, parsed); + } + return map; +} + +function loadRulesCached(): Map { + if (rulesCache === null) { + rulesCache = loadAllRules(); + } + return rulesCache; +} + +// ── Public API ────────────────────────────────────────────────────────────── + +/** Get the interception rules for a single provider, or null if not configured. */ +export function getInterceptionRules(provider: string): ProviderInterceptionRules | null { + return toNormalizedString(provider) ? (loadRulesCached().get(provider) ?? null) : null; +} + +/** Upsert the entire interception rule set for a provider. Invalidates the cache. */ +export function setInterceptionRules(provider: string, rules: ProviderInterceptionRules): void { + const normalizedProvider = toNormalizedString(provider); + if (!normalizedProvider) return; + + const db = getDbInstance(); + const stmt = db.prepare( + "INSERT OR REPLACE INTO key_value (namespace, key, value) VALUES (?, ?, ?)" + ); + + const normalized: ProviderInterceptionRules = { + interceptSearch: rules.interceptSearch, + interceptFetch: rules.interceptFetch, + fetchBackend: rules.fetchBackend, + fetchProxyUrl: rules.fetchProxyUrl, + models: rules.models && Object.keys(rules.models).length > 0 ? rules.models : undefined, + }; + + stmt.run(NAMESPACE, normalizedProvider, JSON.stringify(normalized)); + invalidateCache(); +} + +/** Delete the interception rules for a provider. Resets that provider to default behavior. */ +export function deleteInterceptionRules(provider: string): void { + const normalizedProvider = toNormalizedString(provider); + if (!normalizedProvider) return; + + const db = getDbInstance(); + db.prepare("DELETE FROM key_value WHERE namespace = ? AND key = ?").run( + NAMESPACE, + normalizedProvider + ); + invalidateCache(); +} + +/** + * Resolve the effective `interceptSearch` override for a provider/model pair. + * + * Precedence: per-model rule > provider-level rule > undefined (no override — the + * caller should fall back to the existing native-bypass defaults). + */ +export function resolveInterceptSearch( + provider: string | null | undefined, + model: string | null | undefined +): boolean | undefined { + const normalizedProvider = toNormalizedString(provider); + if (!normalizedProvider) return undefined; + + const rules = getInterceptionRules(normalizedProvider); + if (!rules) return undefined; + + const normalizedModel = toNormalizedString(model); + if (normalizedModel && rules.models?.[normalizedModel]?.interceptSearch !== undefined) { + return rules.models[normalizedModel].interceptSearch; + } + + return rules.interceptSearch; +} diff --git a/src/lib/db/migrations/120_interception_rules.sql b/src/lib/db/migrations/120_interception_rules.sql new file mode 100644 index 0000000000..d042a5f035 --- /dev/null +++ b/src/lib/db/migrations/120_interception_rules.sql @@ -0,0 +1,16 @@ +-- 120_interception_rules.sql +-- Documents the interception_rules namespace in the key_value table (#3384). +-- No schema change — the key_value table already exists. +-- Key = provider ID, value = JSON with shape: +-- { +-- interceptSearch?: boolean, +-- interceptFetch?: boolean, +-- fetchBackend?: "firecrawl"|"jina"|"tavily", +-- fetchProxyUrl?: string, +-- models?: { [modelId]: { interceptSearch?, interceptFetch?, fetchBackend?, fetchProxyUrl? } } +-- } +-- +-- Resolution precedence: per-model rule > provider-level rule > undefined (caller +-- falls back to the existing native web-search-bypass defaults in webSearchFallback.ts). +-- +-- See: src/lib/db/interceptionRules.ts diff --git a/src/lib/localDb.ts b/src/lib/localDb.ts index 9301182fbb..34468a6cdb 100755 --- a/src/lib/localDb.ts +++ b/src/lib/localDb.ts @@ -788,7 +788,6 @@ export type { // proxy_logs — export query (#3500 slice 4) // --------------------------------------------------------------------------- export { exportProxyLogsSince } from "./db/proxyLogs"; - // --------------------------------------------------------------------------- // Per-connection 429 cooldown wrappers (#5957 / #5958 — Issue 1 follow-ups) // Logic lives in db/providers/rateLimit.ts (Hard Rule #2 — localDb is re-export @@ -797,3 +796,4 @@ export { exportProxyLogsSince } from "./db/proxyLogs"; export { markConnectionRateLimitedUntil, clearConnectionRateLimit } from "./db/providers"; // Provider param filters — denylist/allowlist config per provider/model (#6625) export * from "./db/paramFilters"; +export * from "./db/interceptionRules"; // Per-model web-search/web-fetch interception rules (#3384) diff --git a/src/shared/components/Sidebar.search.test.tsx b/src/shared/components/Sidebar.search.test.tsx new file mode 100644 index 0000000000..281330078f --- /dev/null +++ b/src/shared/components/Sidebar.search.test.tsx @@ -0,0 +1,113 @@ +// @vitest-environment jsdom +import React from "react"; +import { act } from "react"; +import { createRoot, type Root } from "react-dom/client"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; + +// Skip CloudSyncStatus entirely (it polls /api/sync/cloud + uses next/navigation's +// useRouter, which we don't otherwise need to mock for this component). +process.env.NEXT_PUBLIC_OMNIROUTE_E2E_MODE = "1"; + +vi.mock("next-intl", () => ({ + useTranslations: () => { + const translate = (key: string) => key; + translate.has = () => false; + return translate; + }, +})); + +vi.mock("next/navigation", () => ({ + usePathname: () => "/dashboard/combos", +})); + +const cleanupCallbacks: Array<() => void> = []; + +function makeContainer(): HTMLElement { + const container = document.createElement("div"); + document.body.appendChild(container); + cleanupCallbacks.push(() => container.remove()); + return container; +} + +function jsonResponse(body: unknown) { + return { ok: true, status: 200, json: async () => body } as Response; +} + +describe("Sidebar search/filter (#4013)", () => { + let root: Root | undefined; + + beforeEach(() => { + ( + globalThis as typeof globalThis & { IS_REACT_ACT_ENVIRONMENT?: boolean } + ).IS_REACT_ACT_ENVIRONMENT = true; + vi.stubGlobal( + "fetch", + vi.fn(async (url: string) => { + if (String(url).includes("/api/settings")) return jsonResponse({}); + return jsonResponse({}); + }) + ); + }); + + afterEach(() => { + if (root) { + act(() => root!.unmount()); + root = undefined; + } + while (cleanupCallbacks.length > 0) cleanupCallbacks.pop()?.(); + document.body.innerHTML = ""; + vi.unstubAllGlobals(); + vi.resetModules(); + }); + + it("renders a search input at the top of the expanded sidebar", async () => { + // First import in this file pays the one-time cost of compiling Sidebar's + // large transitive dependency graph (sidebarVisibility sections, icons, etc). + const { default: Sidebar } = await import("@/shared/components/Sidebar"); + const container = makeContainer(); + root = createRoot(container); + await act(async () => { + root!.render(); + }); + + const input = container.querySelector('input[type="search"]'); + expect(input).toBeTruthy(); + }, 20000); + + it("filters visible nav items down to those matching the typed query", async () => { + const { default: Sidebar } = await import("@/shared/components/Sidebar"); + const container = makeContainer(); + root = createRoot(container); + await act(async () => { + root!.render(); + }); + + const linksBefore = container.querySelectorAll("nav a"); + expect(linksBefore.length).toBeGreaterThan(1); + + const input = container.querySelector('input[type="search"]') as HTMLInputElement; + expect(input).toBeTruthy(); + + const nativeSetter = Object.getOwnPropertyDescriptor( + HTMLInputElement.prototype, + "value" + )!.set!; + + await act(async () => { + nativeSetter.call(input, "zzz-no-such-nav-item-zzz"); + input.dispatchEvent(new Event("input", { bubbles: true })); + }); + + const linksAfterNoMatch = container.querySelectorAll("nav a"); + expect(linksAfterNoMatch.length).toBe(0); + expect(container.querySelector("nav")?.textContent).toBeTruthy(); + + await act(async () => { + nativeSetter.call(input, ""); + input.dispatchEvent(new Event("input", { bubbles: true })); + }); + + const linksAfterClear = container.querySelectorAll("nav a"); + expect(linksAfterClear.length).toBe(linksBefore.length); + }); +}); diff --git a/src/shared/components/Sidebar.tsx b/src/shared/components/Sidebar.tsx index b72636b79c..ead99f0224 100644 --- a/src/shared/components/Sidebar.tsx +++ b/src/shared/components/Sidebar.tsx @@ -5,9 +5,11 @@ import Link from "next/link"; import { usePathname } from "next/navigation"; import { cn } from "@/shared/utils/cn"; import { getActiveSidebarHref } from "@/shared/utils/sidebarRouteMatch"; +import { filterSidebarSectionsByQuery } from "@/shared/utils/sidebarSearch"; import { APP_CONFIG } from "@/shared/constants/appConfig"; import OmniRouteLogo from "./OmniRouteLogo"; import Button from "./Button"; +import Input from "./Input"; import { ConfirmModal } from "./Modal"; import CloudSyncStatus from "./CloudSyncStatus"; import { useTranslations } from "next-intl"; @@ -101,6 +103,7 @@ export default function Sidebar({ ); const [pinnedSections, setPinnedSections] = useState>(new Set()); const [hoveredItem, setHoveredItem] = useState(null); + const [searchQuery, setSearchQuery] = useState(""); // Load persisted state on mount; OmniProxy is pinned by default on first visit useEffect(() => { @@ -266,6 +269,11 @@ export default function Sidebar({ const activeHref = getActiveSidebarHref(pathname, allVisibleItems); + const isSearching = searchQuery.trim().length > 0; + const displaySections = isSearching + ? filterSidebarSectionsByQuery(visibleSections, searchQuery) + : visibleSections; + // Auto-expand the section containing the active page (without closing others) useEffect(() => { if (collapsed) return; @@ -518,6 +526,21 @@ export default function Sidebar({ + {!collapsed && ( +
+ setSearchQuery(e.target.value)} + placeholder={tc("search")} + aria-label={tc("search")} + icon="search" + className="gap-0" + inputClassName="py-1.5 text-xs" + /> +
+ )} +