mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-09-01 03:52:17 +03:00
Compare commits
2 Commits
chore/esli
...
docs/opena
| Author | SHA1 | Date | |
|---|---|---|---|
|
3113e571fc | ||
|
|
6706c382d8 |
56
AGENTS.md
56
AGENTS.md
@@ -58,7 +58,7 @@ Repository map and Reference Documentation sections below.
|
||||
| Services | `open-sse/services/` | Combo routing, rate limits, caching, etc |
|
||||
| Database | `src/lib/db/` | SQLite domain modules (167 migrations) |
|
||||
| Domain/Policy | `src/domain/` | Policy engine, cost rules, fallback logic |
|
||||
| MCP Server | `open-sse/mcp-server/` | 110 tools (44 canonical + memory/skill/GitHub/pool/gamification/plugin/Notion/Obsidian/local-corpus/RTK modules), 3 transports (stdio / SSE / Streamable HTTP), 33 scopes |
|
||||
| MCP Server | `open-sse/mcp-server/` | 110 tools (45 canonical + memory/skill/GitHub/pool/gamification/plugin/Notion/Obsidian/local-corpus/RTK modules), 3 transports (stdio / SSE / Streamable HTTP), 33 scopes |
|
||||
| A2A Server | `src/lib/a2a/` | JSON-RPC 2.0 agent protocol |
|
||||
| Skills | `src/lib/skills/` | Extensible skill framework |
|
||||
| Memory | `src/lib/memory/` | Persistent conversational memory |
|
||||
@@ -110,26 +110,32 @@ upstream/service level, so one unhealthy provider does not slow down every reque
|
||||
- Shared wrappers: `open-sse/services/accountFallback.ts`
|
||||
- Persisted state table: `domain_circuit_breakers`
|
||||
|
||||
**States**:
|
||||
**States** (4 — `src/shared/utils/circuitBreaker.ts`):
|
||||
|
||||
- `CLOSED`: normal traffic is allowed.
|
||||
- `DEGRADED`: early-warning band — failures crossed the degradation threshold but not the
|
||||
breaker threshold yet; traffic still flows, dashboards show the warning.
|
||||
- `OPEN`: provider is temporarily blocked; callers get a provider-circuit-open response
|
||||
or combo routing skips to another target.
|
||||
- `HALF_OPEN`: reset timeout has elapsed; allow a probe request. Success closes the
|
||||
breaker, failure opens it again.
|
||||
|
||||
**Defaults** (`open-sse/config/constants.ts` → `PROVIDER_PROFILES`). Two thresholds live side by
|
||||
side — do not confuse them:
|
||||
**Defaults** (`open-sse/config/constants.ts` → `PROVIDER_PROFILES`, consumed via
|
||||
`DEFAULT_RESILIENCE_SETTINGS.providerBreaker` in `src/lib/resilience/settings.ts` →
|
||||
`getCircuitBreaker(provider, …)` in `src/sse/handlers/chatHelpers.ts`). The whole-provider
|
||||
breaker runs on `circuitBreakerThreshold` / `circuitBreakerReset`:
|
||||
|
||||
| Profile | `providerFailureThreshold` (whole provider) | `providerCooldownMs` | `circuitBreakerThreshold` (one connection) | `circuitBreakerReset` |
|
||||
| ------- | ------------------------------------------: | -------------------: | -----------------------------------------: | --------------------: |
|
||||
| OAuth | `10` | `5min` | `8` | `60s` |
|
||||
| API key | `15` | `10min` | `12` | `30s` |
|
||||
| Local | `2` | `1min` | `2` | `15s` |
|
||||
| Profile | degrades at | opens at (`circuitBreakerThreshold`) | reset (`circuitBreakerReset`) |
|
||||
| ------- | ----------: | -----------------------------------: | ----------------------------: |
|
||||
| OAuth | `5` | `8` | `60s` |
|
||||
| API key | `7` | `12` | `30s` |
|
||||
| Local | (derived) | `2` | `15s` |
|
||||
|
||||
The provider-level thresholds were scaled up for deployments with 500+ connections (OAuth was
|
||||
`3`, API key was `5`); every default is overridable through the `OMNIROUTE_PROVIDER_BREAKER_*`
|
||||
and `OMNIROUTE_CIRCUIT_BREAKER_*` env vars.
|
||||
`PROVIDER_PROFILES` also defines `providerFailureThreshold` (10/15/2) and `providerCooldownMs`
|
||||
(5min/10min/1min); those fields are loaded into the profile but have **no runtime consumer
|
||||
today** — do not tune or document them as the live breaker. Every default is overridable
|
||||
through the `OMNIROUTE_PROVIDER_BREAKER_*` and `OMNIROUTE_CIRCUIT_BREAKER_*` env vars; the
|
||||
runtime-accurate reference table lives in `docs/architecture/RESILIENCE_GUIDE.md`.
|
||||
|
||||
Only provider-level failure statuses should trip the provider breaker:
|
||||
|
||||
@@ -242,7 +248,7 @@ Read the nearest `AGENTS.md` and the linked deep-dive before making a non-trivia
|
||||
| Streaming request handling | `open-sse/handlers/` | [`docs/architecture/ARCHITECTURE.md`](docs/architecture/ARCHITECTURE.md) |
|
||||
| Provider execution and translation | `open-sse/executors/`, `open-sse/translator/` | [`docs/architecture/CODEBASE_DOCUMENTATION.md`](docs/architecture/CODEBASE_DOCUMENTATION.md) |
|
||||
| Routing and resilience | `open-sse/services/` | [`open-sse/services/AGENTS.md`](open-sse/services/AGENTS.md), [`docs/routing/AUTO-COMBO.md`](docs/routing/AUTO-COMBO.md) |
|
||||
| Database and migrations | `src/lib/db/`, `db/migrations/` | [`src/lib/db/AGENTS.md`](src/lib/db/AGENTS.md) |
|
||||
| Database and migrations | `src/lib/db/`, `src/lib/db/migrations/` | [`src/lib/db/AGENTS.md`](src/lib/db/AGENTS.md) |
|
||||
| Domain policy | `src/domain/` | [`docs/architecture/ARCHITECTURE.md`](docs/architecture/ARCHITECTURE.md) |
|
||||
| MCP and A2A | `open-sse/mcp-server/`, `src/lib/a2a/` | [`docs/frameworks/MCP-SERVER.md`](docs/frameworks/MCP-SERVER.md), [`docs/frameworks/A2A-SERVER.md`](docs/frameworks/A2A-SERVER.md) |
|
||||
| Agent features | `src/lib/{acp,memory,skills,cloudAgent}/` | [`docs/frameworks/AGENT_PROTOCOLS_GUIDE.md`](docs/frameworks/AGENT_PROTOCOLS_GUIDE.md), [`docs/frameworks/SKILLS.md`](docs/frameworks/SKILLS.md) |
|
||||
@@ -254,13 +260,13 @@ Read the nearest `AGENTS.md` and the linked deep-dive before making a non-trivia
|
||||
## File placement & repo-root hygiene
|
||||
|
||||
- **Test files**: ALL unit tests, integration tests, ecosystem tests, or Vitest files MUST strictly be placed within the `tests/` directory (e.g., `tests/unit/`, `tests/integration/`). NEVER create test files in the project root (`/`).
|
||||
- **Scripts and utilities**: ALL maintenance, debugging, generation, or experimental scripts (`.cjs`, `.mjs`, `.js`, `.ts`) MUST be placed strictly inside one of the `scripts/` subfolders (`build/`, `dev/`, `check/`, `docs/`, `i18n/`, `ad-hoc/`, `quality/`, `release/`, `ci/`, `ops/`, `perf/`, `research/`, `sre/`, `vps/`, `homolog/`, `raycast/`, `skills/`, `test/`, `cli/`, `compression/`, `compression-eval/`, `devin-bridge/`, `docker/`, `features/`, `router-eval/`). One-shot or experimental code goes under `scripts/ad-hoc/`. NEVER dump loose scripts in the project root (`/`) or the top-level `scripts/` folder.
|
||||
- **Scripts and utilities**: ALL maintenance, debugging, generation, or experimental scripts (`.cjs`, `.mjs`, `.js`, `.ts`) MUST be placed strictly inside one of the `scripts/` subfolders (`build/`, `dev/`, `check/`, `docs/`, `i18n/`, `ad-hoc/`, `quality/`, `release/`, `ci/`, `ops/`, `perf/`, `research/`, `sre/`, `vps/`, `homolog/`, `packs/`, `skills/`, `test/`, `cli/`, `compression/`, `compression-eval/`, `devin-bridge/`, `docker/`, `features/`, `router-eval/`). One-shot or experimental code goes under `scripts/ad-hoc/`. NEVER dump loose scripts in the project root (`/`) or the top-level `scripts/` folder.
|
||||
|
||||
**The project root MUST ONLY contain:**
|
||||
|
||||
- Configuration files (`vitest.config.ts`, `next.config.mjs`, `eslint.config.mjs`, `tsconfig*.json`, `playwright.config.ts`, `prettier.config.mjs`, `postcss.config.mjs`, `sonar-project.properties`, `fly.toml`, `docker-compose*.yml`, `Dockerfile`)
|
||||
- Dependency files (`package.json`, `package-lock.json`)
|
||||
- Documentation files (`README.md`, `CHANGELOG.md`, `ROADMAP.md`, `LICENSE`, `AGENTS.md`, `CLAUDE.md`, `GEMINI.md`, `CONTRIBUTING.md`, `SECURITY.md`, `CODE_OF_CONDUCT.md`, `llm.txt`, `Tuto_Qdrant.md`)
|
||||
- Documentation files (`README.md`, `CHANGELOG.md`, `ROADMAP.md`, `LICENSE`, `AGENTS.md`, `CLAUDE.md`, `GEMINI.md`, `CONTRIBUTING.md`, `SECURITY.md`, `CODE_OF_CONDUCT.md`, `llm.txt`)
|
||||
- CI/CD files and ignore definitions (`.gitignore`, `.dockerignore`, `.npmignore`, `.npmrc`, `.node-version`, `.nvmrc`, `.env.example`)
|
||||
|
||||
When creating _any_ validation tests or one-off logic scripts, default to `scripts/ad-hoc/` or `tests/unit/` according to your goals. Do not pollute the `/` root context.
|
||||
@@ -361,11 +367,11 @@ Documentation must describe verified behavior, not plausible behavior.
|
||||
1. Add tool definition in `open-sse/mcp-server/tools/` with Zod input schema + async handler
|
||||
2. Register in tool set (wired by `createMcpServer()`)
|
||||
3. Assign to appropriate scope(s)
|
||||
4. Write tests (tool invocation logged to `mcp_audit` table)
|
||||
4. Write tests (tool invocation logged to the `mcp_tool_audit` table)
|
||||
|
||||
### Adding a New A2A Skill
|
||||
|
||||
1. Create skill in `src/lib/a2a/skills/` (5 already exist: smart-routing, quota-management, provider-discovery, cost-analysis, health-report)
|
||||
1. Create skill in `src/lib/a2a/skills/` (6 already exist: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
2. Skill receives task context (messages, metadata) → returns structured result
|
||||
3. Register in `A2A_SKILL_HANDLERS` in `src/lib/a2a/taskExecution.ts`
|
||||
4. Expose in `src/app/.well-known/agent.json/route.ts` (Agent Card)
|
||||
@@ -374,7 +380,7 @@ Documentation must describe verified behavior, not plausible behavior.
|
||||
|
||||
### Adding a New Cloud Agent
|
||||
|
||||
1. Create agent class in `src/lib/cloudAgent/agents/` extending `CloudAgentBase` (3 already exist: codex-cloud, devin, jules)
|
||||
1. Create agent class in `src/lib/cloudAgent/agents/` extending `CloudAgentBase` (4 already exist: codex-cloud, devin, jules, cursor-cloud)
|
||||
2. Implement `createTask`, `getStatus`, `approvePlan`, `sendMessage`, `listSources`
|
||||
3. Register in `src/lib/cloudAgent/registry.ts`
|
||||
4. Add OAuth/credentials handling if needed (`src/lib/oauth/providers/`)
|
||||
@@ -385,7 +391,7 @@ Documentation must describe verified behavior, not plausible behavior.
|
||||
1. Create installer in `src/lib/services/installers/{name}.ts` modeled on `ninerouter.ts` (use `runNpm` from `installers/utils.ts` — no shell interpolation, hard rule #13).
|
||||
2. Register the service in `src/lib/services/bootstrap.ts` (add to `SERVICES[]` array and extend `buildSpawnArgsFactory()`).
|
||||
3. Add a DB seed row for the new service in `src/lib/db/migrations/` (`version_manager` table, `status='not_installed'`, `auto_start=0`).
|
||||
4. Create 7 API endpoints under `src/app/api/services/{name}/` (`_lib.ts`, `install`, `start`, `stop`, `restart`, `update`, `status`, `auto-start`). All delegate errors through `createErrorResponse()`. The shared `logs` endpoint is already wired via `[name]/logs/route.ts`.
|
||||
4. Create 8 API endpoints under `src/app/api/services/{name}/` (`_lib.ts`, `install`, `start`, `stop`, `restart`, `update`, `status`, `auto-start`, `auto-restart-adopted`). All delegate errors through `createErrorResponse()`. The shared `logs` endpoint is already wired via `[name]/logs/route.ts`.
|
||||
5. Verify `/api/services/` is in `LOCAL_ONLY_API_PREFIXES` in `src/server/authz/routeGuard.ts`; add a test asserting `isLocalOnlyPath()` returns `true` for the new prefix if you add one (hard rule #17).
|
||||
6. Add a UI tab in `src/app/(dashboard)/dashboard/providers/services/tabs/` reusing `ServiceStatusCard`, `ServiceLifecycleButtons`, `ServiceLogsPanel`.
|
||||
7. Document in `docs/frameworks/EMBEDDED-SERVICES.md` (update §1 service table + §4 API reference) and `docs/openapi.yaml`.
|
||||
@@ -438,7 +444,7 @@ For any non-trivial change, read the matching deep-dive first:
|
||||
| VS Code Copilot Chat (OmniCopilot extension) | `docs/guides/VSCODE-COPILOT.md` |
|
||||
| Release flow | `docs/ops/RELEASE_CHECKLIST.md` |
|
||||
| Embedded services | `docs/frameworks/EMBEDDED-SERVICES.md` |
|
||||
| Quality gates (~80 scripts, allowlist policy) | `docs/architecture/QUALITY_GATES.md` |
|
||||
| Quality gates (~90 scripts, allowlist policy) | `docs/architecture/QUALITY_GATES.md` |
|
||||
|
||||
---
|
||||
|
||||
@@ -629,8 +635,8 @@ focused checks, and use a Conventional Commit message (for example, `docs: slim
|
||||
|
||||
## Environment
|
||||
|
||||
- **Runtime**: Node.js ≥22.0.0 <23 || ≥24.0.0 <27, ES Modules. This is the **only supported** runtime for the published `omniroute` CLI, the server, and the test suites (`node:test` + vitest) — `engines.node` is authoritative and end users never need Bun. A **best-effort `bun:sqlite` compatibility path** exists so a global Bun install (`bun install -g omniroute`) can start without `better-sqlite3` (driver adapter + Bun-aware process spawning); it is **not** a supported runtime — no support guarantees — and every Bun-specific runtime change MUST preserve the Node driver/fallback chain and ship a Bun test (`test:bun:db`) or an explicit reason why the path is Node-only.
|
||||
- **Bun (build/dev script runner + compatibility smoke only)**: Bun `1.3.14` is pinned as an **exact devDependency** (provisioned through the existing `npm ci` via the lockfile's `@oven/bun-*` platform binaries — no `setup-bun`/ad-hoc install). It is used **only** to execute a small, allow-listed set of TypeScript **gate/generator scripts** (replacing `node --import tsx` for startup speed): the CI checks `check:provider-consistency`, `check:compression-budget`, `check:known-symbols`, and the non-CI `gen:provider-reference`, `bench:compression` — plus the focused `test:bun:db` compatibility smoke suite for the best-effort `bun:sqlite` path. **Do NOT** widen Bun to `npm install`, the build (`build:cli*`), `check:pack-artifact`, the supported published runtime, or the main test runners — those stay on Node. Any new Bun-invoking gate/generator script must be validated byte-identical against its `node --import tsx` output first. After pulling the lockfile change, run `npm install` so `bun` resolves locally (a stale `node_modules` will fail those scripts with `bun: not found`).
|
||||
- **Runtime**: Node.js ≥22.22.2 <23 || ≥24.0.0 <27, ES Modules. This is the **only supported** runtime for the published `omniroute` CLI, the server, and the test suites (`node:test` + vitest) — `engines.node` is authoritative and end users never need Bun. A **best-effort `bun:sqlite` compatibility path** exists so a global Bun install (`bun install -g omniroute`) can start without `better-sqlite3` (driver adapter + Bun-aware process spawning); it is **not** a supported runtime — no support guarantees — and every Bun-specific runtime change MUST preserve the Node driver/fallback chain and ship a Bun test (`test:bun:db`) or an explicit reason why the path is Node-only.
|
||||
- **Bun (build/dev script runner + compatibility smoke only)**: Bun `1.4.0` is pinned as an **exact devDependency** (provisioned through the existing `npm ci` via the lockfile's `@oven/bun-*` platform binaries — no `setup-bun`/ad-hoc install). It is used **only** to execute a small, allow-listed set of TypeScript **gate/generator scripts** (replacing `node --import tsx` for startup speed): the CI checks `check:provider-consistency`, `check:compression-budget`, `check:known-symbols`, and the non-CI `gen:provider-reference`, `bench:compression` — plus the focused `test:bun:db` compatibility smoke suite for the best-effort `bun:sqlite` path. **Do NOT** widen Bun to `npm install`, the build (`build:cli*`), `check:pack-artifact`, the supported published runtime, or the main test runners — those stay on Node. Any new Bun-invoking gate/generator script must be validated byte-identical against its `node --import tsx` output first. After pulling the lockfile change, run `npm install` so `bun` resolves locally (a stale `node_modules` will fail those scripts with `bun: not found`).
|
||||
- **TypeScript**: 6.0+, target ES2022, module esnext, resolution bundler
|
||||
- **Path aliases**: `@/*` → `src/`, `@omniroute/open-sse` → `open-sse/`, `@omniroute/open-sse/*` → `open-sse/*`
|
||||
- **Default port**: 20128 (API + dashboard on same port)
|
||||
@@ -642,12 +648,12 @@ focused checks, and use a Conventional Commit message (for example, `docs: slim
|
||||
|
||||
## Quality Gates & Ratchets
|
||||
|
||||
OmniRoute has **~80 quality-gate scripts** (`scripts/check/` + `scripts/quality/`) wired
|
||||
OmniRoute has **~90 quality-gate scripts** (`scripts/check/` + `scripts/quality/`) wired
|
||||
across **9 gate-running jobs** in `.github/workflows/ci.yml` (`lint`, `quality-gate`,
|
||||
`quality-extended`, `docs-sync-strict`, `i18n-ui-coverage`, `i18n`, `pr-test-policy`,
|
||||
`test-vitest`, `sonarqube`), plus the `quality.yml` fast-gates job (PR→`release/**`) and
|
||||
3 nightly workflows (`nightly-property`, `nightly-resilience`, `nightly-llm-security`;
|
||||
`nightly-mutation` once merged). Full inventory, per-job breakdown, and operational
|
||||
5 quality nightly workflows (`nightly-property`, `nightly-resilience`,
|
||||
`nightly-llm-security`, `nightly-mutation`, `nightly-schemathesis`). Full inventory, per-job breakdown, and operational
|
||||
procedures are in [`docs/architecture/QUALITY_GATES.md`](docs/architecture/QUALITY_GATES.md).
|
||||
|
||||
**Quick reference:**
|
||||
|
||||
@@ -301,7 +301,7 @@ src/ # TypeScript (.ts / .tsx)
|
||||
open-sse/ # @omniroute/open-sse workspace
|
||||
├── executors/ # 89 executor implementation modules
|
||||
├── handlers/ # 11 request handlers (chat, responses, embeddings, images, etc.)
|
||||
├── mcp-server/ # MCP server (107 unique tools, 3 transports, 32 scopes)
|
||||
├── mcp-server/ # MCP server (110 unique tools, 3 transports, 33 scopes)
|
||||
├── services/ # 178 top-level services (combo, autoCombo, rateLimitManager, etc.)
|
||||
├── translator/ # Format translators (OpenAI ↔ Claude ↔ Gemini ↔ Responses ↔ Ollama)
|
||||
├── transformer/ # Responses API transformer
|
||||
|
||||
41
README.md
41
README.md
@@ -7,7 +7,7 @@
|
||||
|
||||
# 🚀 OmniRoute — The Free AI Gateway
|
||||
|
||||
<img src="./docs/diagrams/readme-hero.svg" width="100%" alt="OmniRoute — Never stop coding. Every AI tool → 352 providers — 90+ free — through one endpoint. Claude Code, Codex, Cursor, Cline, Copilot & Antigravity into FREE Claude / GPT / Gemini with auto-fallback. RTK + Caveman stacked compression saves 15–95% tokens (~89% avg) — never hit limits. 352 AI providers · 90+ free tiers · ~1.51B free tokens/mo · 19 routing strategies · $0 to start."/>
|
||||
<img src="./docs/diagrams/readme-hero.svg" width="100%" alt="OmniRoute — Never stop coding. Every AI tool → 352 providers — 150+ free — through one endpoint. Claude Code, Codex, Cursor, Cline, Copilot & Antigravity into FREE Claude / GPT / Gemini with auto-fallback. RTK + Caveman stacked compression saves 15–95% tokens (~89% avg) — never hit limits. 352 AI providers · 150+ free tiers · ~1.51B free tokens/mo · 19 routing strategies · $0 to start."/>
|
||||
|
||||
</div>
|
||||
|
||||
@@ -17,9 +17,9 @@
|
||||
|
||||
</div>
|
||||
|
||||
> Stacking free tiers by hand is painful — dozens of SDKs, dozens of rate limits, and no idea how much you actually have. OmniRoute catalogs **445 free-tier entries across 39 recurring pool keys** and computes the token headline from the **20 pools with a published positive monthly budget**, deduplicated by shared pool. The result stays visible on the dashboard (`/dashboard/free-tiers`).
|
||||
> Stacking free tiers by hand is painful — dozens of SDKs, dozens of rate limits, and no idea how much you actually have. OmniRoute catalogs **446 free-tier entries across 38 recurring pool keys** and computes the token headline from the **20 pools with a published positive monthly budget**, deduplicated by shared pool. The result stays visible on the dashboard (`/dashboard/free-tiers`).
|
||||
|
||||
<img src="./docs/diagrams/free-tier-budget.svg" width="100%" alt="OmniRoute free-tier budget card: ~1.51B free tokens per month steady, up to ~2.13B in the first month with signup credits, from 39 documented recurring pool keys covering 445 cataloged free-tier entries behind one endpoint. Honest pool-deduped math — each shared pool counted once, including 20 recurring pools with a published positive monthly token budget; 13 providers are marked avoid in the terms-risk catalog so you decide. Budget bar includes Mistral 1B, LLM7 150M, Nara 150M, Gemini 60M and smaller pools, plus first-month signup credits and permanently-free no-token-cap providers surfaced separately so they never inflate the headline. Live used/remaining on /dashboard/free-tiers."/>
|
||||
<img src="./docs/diagrams/free-tier-budget.svg" width="100%" alt="OmniRoute free-tier budget card: ~1.51B free tokens per month steady, up to ~2.13B in the first month with signup credits, from 38 documented recurring pool keys covering 446 cataloged free-tier entries behind one endpoint. Honest pool-deduped math — each shared pool counted once, including 20 recurring pools with a published positive monthly token budget; 13 providers are marked avoid in the terms-risk catalog so you decide. Budget bar includes Mistral 1B, LLM7 150M, Nara 150M, Gemini 60M and smaller pools, plus first-month signup credits and permanently-free no-token-cap providers surfaced separately so they never inflate the headline. Live used/remaining on /dashboard/free-tiers."/>
|
||||
|
||||
> Animated summary of the live `/dashboard/free-tiers` page. Full methodology (pool dedupe, credit tiers, provider terms): **[docs/reference/FREE_TIERS.md](docs/reference/FREE_TIERS.md)**.
|
||||
>
|
||||
@@ -63,7 +63,7 @@
|
||||
|
||||
| | v3.8.49 | **v3.8.50** | `v3.8.51+` |
|
||||
| ------------------------- | :-----: | :-----------------------: | :---------: |
|
||||
| 🌐 Providers | 290 | **350** | more queued |
|
||||
| 🌐 Providers | 290 | **352** | more queued |
|
||||
| 🧠 Unique chat model IDs | 1185 | **1312** | — |
|
||||
| 🖼️ Modality Bridge | — | 🆕 vision + audio + video | — |
|
||||
| 📡 Radar free catalog | — | 🆕 opt-in | — |
|
||||
@@ -210,7 +210,7 @@ curl http://localhost:20128/v1/chat/completions \
|
||||
|
||||
</div>
|
||||
|
||||
<img src="./docs/diagrams/promise-pillars.svg" width="100%" alt="The Promise — One endpoint and 352 providers. Automatic fallback keeps routing while another healthy target is available. Six pillars: resilient fallback across 352 providers · up to 95% token savings on eligible workloads · $0 to start with 90+ free tiers and 56 recurring/keyless free-forever providers · 35 CLI/agent integrations through one config · OpenAI, Claude, Gemini and Responses API compatibility at /v1 · production controls including circuit breakers, TLS stealth, MCP 110 tools, A2A, memory, guardrails, evals and 39,000+ static test declarations across 5,100+ tracked test files."/>
|
||||
<img src="./docs/diagrams/promise-pillars.svg" width="100%" alt="The Promise — One endpoint and 352 providers. Automatic fallback keeps routing while another healthy target is available. Six pillars: resilient fallback across 352 providers · up to 95% token savings on eligible workloads · $0 to start with 150+ free tiers and 53 recurring/keyless free-forever providers · 36 CLI/agent integrations through one config · OpenAI, Claude, Gemini and Responses API compatibility at /v1 · production controls including circuit breakers, TLS stealth, MCP 110 tools, A2A, memory, guardrails, evals and 39,000+ static test declarations across 5,100+ tracked test files."/>
|
||||
|
||||
<br/>
|
||||
<br/>
|
||||
@@ -332,6 +332,8 @@ No combo to create. Set your model to `auto` (or a variant) and OmniRoute builds
|
||||
<tr><td align="left" nowrap><code>auto/cheap</code></td><td align="left">💰 Cheapest per token first</td></tr>
|
||||
<tr><td align="left" nowrap><code>auto/offline</code></td><td align="left">🔋 Most quota / rate-limit headroom first</td></tr>
|
||||
<tr><td align="left" nowrap><code>auto/smart</code></td><td align="left">🔭 Quality-first + 10% exploration to discover better models</td></tr>
|
||||
<tr><td align="left" nowrap><code>auto/lkgp</code></td><td align="left">📌 Explicit last-known-good-provider stickiness</td></tr>
|
||||
<tr><td align="left" nowrap><code>auto/chaos</code></td><td align="left">🧪 Fault-injection weights for resilience testing (chaos engineering)</td></tr>
|
||||
</table>
|
||||
|
||||
##
|
||||
@@ -449,7 +451,7 @@ All **19** strategies — mix & match per combo step:
|
||||
|
||||
### 🧱 Resilience is built in (3 independent layers)
|
||||
|
||||
<img src="./docs/diagrams/resilience-layers.svg" width="100%" alt="OmniRoute resilience — 3 independent self-healing layers, the right layer for the right failure. Layer 1 provider circuit breaker (whole provider): trips only on 408/5xx, thresholds OAuth 10× / API-key 15× / local 2×, resets 60s/30s/15s into a HALF-OPEN probe, lazy recovery; while OPEN the combo reroutes to the next provider. Layer 2 connection cooldown (one key/account): base 5s OAuth / 3s API-key, exponential ×2 backoff with anti-thundering-herd guard, 429 honors Retry-After, success clears all error state; one cooling key is skipped while sibling keys keep serving. Layer 3 model lockout (one model): per-model 429, local 404 or mode denials lock just that model — never the whole connection. Terminal states (banned, expired, credits exhausted) are for the operator, not cooldowns."/>
|
||||
<img src="./docs/diagrams/resilience-layers.svg" width="100%" alt="OmniRoute resilience — 3 independent self-healing layers, the right layer for the right failure. Layer 1 provider circuit breaker (whole provider): trips only on 408/5xx, thresholds OAuth 8× / API-key 12× / local 2×, resets 60s/30s/15s into a HALF-OPEN probe, lazy recovery; while OPEN the combo reroutes to the next provider. Layer 2 connection cooldown (one key/account): base 5s OAuth / 3s API-key, exponential ×2 backoff with anti-thundering-herd guard, 429 honors Retry-After, success clears all error state; one cooling key is skipped while sibling keys keep serving. Layer 3 model lockout (one model): per-model 429, local 404 or mode denials lock just that model — never the whole connection. Terminal states (banned, expired, credits exhausted) are for the operator, not cooldowns."/>
|
||||
|
||||
<sub>📖 [Auto-Combo Engine](docs/routing/AUTO-COMBO.md) · [Resilience Guide](docs/architecture/RESILIENCE_GUIDE.md)</sub>
|
||||
|
||||
@@ -461,7 +463,7 @@ All **19** strategies — mix & match per combo step:
|
||||
|
||||
</div>
|
||||
|
||||
<img src="./docs/diagrams/comparison-table.svg" width="100%" alt="What sets OmniRoute apart — a dated feature snapshot vs 9router, OpenRouter, CLIProxyAPI and LiteLLM across 13 capabilities. OmniRoute: 352 providers, 90+ free tiers built in, 19 routing strategies, 12-engine token compression, built-in MCP server with 110 tools, A2A agent protocol, persistent memory, guardrails, cloud agents, TLS fingerprint stealth, Desktop/Termux/PWA and 43 i18n UI locales. OmniRoute is MIT-licensed and self-hostable. Competitor capabilities and counts may change; see the linked methodology."/>
|
||||
<img src="./docs/diagrams/comparison-table.svg" width="100%" alt="What sets OmniRoute apart — a dated feature snapshot vs 9router, OpenRouter, CLIProxyAPI and LiteLLM across 13 capabilities. OmniRoute: 352 providers, 150+ free tiers built in, 19 routing strategies, 12-engine token compression, built-in MCP server with 110 tools, A2A agent protocol, persistent memory, guardrails, cloud agents, TLS fingerprint stealth, Desktop/Termux/PWA and 43 i18n UI locales. OmniRoute is MIT-licensed and self-hostable. Competitor capabilities and counts may change; see the linked methodology."/>
|
||||
|
||||
<sub>📊 Full methodology & per-feature detail vs 9router, OpenRouter, CLIProxyAPI & LiteLLM → [`docs/comparison/OMNIROUTE_VS_ALTERNATIVES.md`](docs/comparison/OMNIROUTE_VS_ALTERNATIVES.md)</sub>
|
||||
|
||||
@@ -548,7 +550,7 @@ the current catalog at **[radar.omniroute.online/planos](https://radar.omniroute
|
||||
- **🗜️ Compression hardening** — default-on inflation guard, Caveman packs for DE / FR / JA + Chinese (wényán), RTK filters for Gradle & .NET. → [Compression](docs/compression/COMPRESSION_ENGINES.md)
|
||||
- **💸 Honest flat-rate cost** — subscription / coding-plan providers read **$0** in cost analytics; budget, quota & routing keep estimating. → [API Reference](docs/reference/API_REFERENCE.md)
|
||||
- **⚖️ Quota-Share routing** — split a shared account's quota fairly across pooled keys, work-conserving so idle slices are lent out. → [Resilience Guide](docs/architecture/RESILIENCE_GUIDE.md)
|
||||
- **🤖 One-command CLI/agent setup** — 12 registered `setup-*` commands; `omniroute run` launches 7 CLIs (Claude Code, Codex, Aider, Goose, OpenCode, Qwen Code, Gemini CLI); `omniroute configure` supports 9 targets with an interactive provider+model picker and per-context favorites. → [CLI Integrations](docs/guides/CLI-INTEGRATIONS.md)
|
||||
- **🤖 One-command CLI/agent setup** — 13 registered `setup-*` commands; `omniroute run` launches 7 CLIs (Claude Code, Codex, Aider, Goose, OpenCode, Qwen Code, Gemini CLI); `omniroute configure` supports 10 targets with an interactive provider+model picker and per-context favorites. → [CLI Integrations](docs/guides/CLI-INTEGRATIONS.md)
|
||||
- **🛰️ Remote mode** — drive a remote OmniRoute with scoped tokens (`connect` / `contexts` / `tokens`) + an `antigravity` OAuth helper for VPS installs. → [Remote Mode](docs/guides/REMOTE-MODE.md)
|
||||
- **🧭 Smarter auto-routing** — `auto/<category>:<tier>` combos, **Fusion** (model panel + judge), task-aware routing, per-request model / mode / USD-budget overrides. → [Auto-Combo](docs/routing/AUTO-COMBO.md)
|
||||
- **🗜️ Pluggable compression** — 12 composable engines + Compression Studios: LLMLingua-2, two-tier Ultra, omniglyph, per-step fidelity gate, GCF v3.2, drag-reorder editor. → [Compression](docs/compression/COMPRESSION_ENGINES.md)
|
||||
@@ -559,9 +561,10 @@ the current catalog at **[radar.omniroute.online/planos](https://radar.omniroute
|
||||
- **🖼️ New endpoints** — `/v1/ocr` (Mistral OCR) and `/v1/audio/translations` (Whisper-style) round out the media surface. → [API Reference](docs/reference/API_REFERENCE.md)
|
||||
- **🎨 Image / video / audio generation** — one API for media: xAI Grok Imagine & Novita AI video, ComfyUI, Magnific, Adobe Firefly, Segmind, and speech providers such as ElevenLabs. → [API Reference](docs/reference/API_REFERENCE.md)
|
||||
- **🌍 Deployment & ops** — reverse-proxy `basePath`, browser-language auto-detect, per-key device tracking, root-less MITM trust, zh-TW localization. → [Environment](docs/reference/ENVIRONMENT.md)
|
||||
- **🤝 More providers & agents** — Cursor Cloud Agent, Grok Build (xAI) with browser + OAuth login, Ollama first-class card, Claude Opus 5 & Sonnet 5, Kimi official partnership (Code/Web/Moonshot), Zed, Requesty, SenseNova, Yuanbao, Agnes AI… and a refreshed **351-provider catalog**. → [Providers](docs/reference/PROVIDER_REFERENCE.md)
|
||||
- **🤝 More providers & agents** — cloud agents (Codex Cloud, Cursor, Devin, Jules), Grok Build (xAI) with browser + OAuth login, Ollama first-class card, Claude Opus 5 & Sonnet 5, Kimi official partnership (Code/Web/Moonshot), Zed, Requesty, SenseNova, Yuanbao, Agnes AI… and a refreshed **352-provider catalog**. → [Providers](docs/reference/PROVIDER_REFERENCE.md)
|
||||
- **📡 Routing transparency** — every response carries an `X-OmniRoute-Decision` header naming the strategy/provider/latency that served it, a new `cache-optimized` combo strategy + Auto-Combo `cacheAffinity` factor route repeat requests back to the connection holding the cached prefix, and a read-only `/v1/auto-combo/{channel}/candidates` endpoint exposes an `auto/*` channel's live candidate pool. → [Auto-Combo](docs/routing/AUTO-COMBO.md)
|
||||
- **⚡ Local performance & infra** — one-click local Redis, Cloudflare Workers / Deno Deploy relay deployers, Bifrost & Mux as supervised embedded services. → [Embedded Services](docs/frameworks/EMBEDDED-SERVICES.md)
|
||||
- **🧩 Also in the box** — plugin framework + marketplace, Omni/Agent/GitHub skills frameworks, Obsidian vault integration (22 MCP tools), OpenAI-compatible Batch & Files APIs, semantic response cache, gamification with leaderboards, ACP agent discovery (15 built-in agents), scheduled log export to BigQuery, `auto/chaos` fault injection, a Telegram bot bridge, an in-app version manager and LMArena-ELO free-provider rankings. → [Docs](docs/README.md)
|
||||
|
||||
<br/>
|
||||
|
||||
@@ -631,7 +634,7 @@ omniroute run qwen --model glm/glm-5.2 -- -p "reply OK"
|
||||
omniroute run gemini --model glm/glm-5.2 -- --skip-trust -p "reply OK"
|
||||
|
||||
# Or pick provider+model interactively and write the tool's own config:
|
||||
omniroute configure codex # also: claude opencode qwen aider goose cline continue kilo
|
||||
omniroute configure codex # also: claude opencode qwen aider goose gemini cline continue kilo
|
||||
```
|
||||
|
||||
Every command honors the active remote context (`omniroute connect <host>`), `--dry-run`
|
||||
@@ -642,11 +645,11 @@ of your shell history. → [CLI Integrations](docs/guides/CLI-INTEGRATIONS.md)
|
||||
|
||||
<div align="center">
|
||||
|
||||
## 🌐 352 AI Providers — 154 Catalog-Marked Free
|
||||
## 🌐 352 AI Providers — 152 Catalog-Marked Free
|
||||
|
||||
</div>
|
||||
|
||||
> **351 registered providers** across the canonical chat, media, search, local, cloud-agent and system collections, including **154 carrying `hasFree: true` discovery metadata**. The chat model registry covers **268 providers / 2,566 distinct provider-model pairs / 1,312 raw model IDs**; the separate free-budget catalog has **455 per-model rows**, **40 recurring pools** and **56 recurring/keyless free-forever providers**. These are different denominators by design; definitions and pool-deduped calculations live in the [Provider Reference](docs/reference/PROVIDER_REFERENCE.md) and [Free Tiers](docs/reference/FREE_TIERS.md).
|
||||
> **352 registered providers** across the canonical chat, media, search, local, cloud-agent and system collections, including **152 carrying `hasFree: true` discovery metadata**. The chat model registry covers **229 providers / 2,554 distinct provider-model pairs / 1,283 raw model IDs**; the separate free-budget catalog has **446 per-model rows**, **38 recurring pools** and **53 recurring/keyless free-forever providers**. These are different denominators by design; definitions and pool-deduped calculations live in the [Provider Reference](docs/reference/PROVIDER_REFERENCE.md) and [Free Tiers](docs/reference/FREE_TIERS.md).
|
||||
|
||||
<div align="center">
|
||||
|
||||
@@ -810,7 +813,7 @@ Tokens are scoped `read` / `write` / `admin`; process-spawning routes stay loopb
|
||||
|
||||
<div align="left">
|
||||
|
||||
<img src="./docs/diagrams/cli-terminal.svg" width="50%" alt="Animated terminal demoing the OmniRoute CLI — omniroute providers list, omniroute combo list and omniroute health — cycling over the 85-command top-level surface: providers · oauth · keys · combo · nodes · models · cache · compression · cost · usage · quota · health · resilience · telemetry · logs · audit · mcp · a2a · cloud · memory · skills · eval · tunnel · backup · sync · webhooks · policy · pricing · translator · simulate …"/>
|
||||
<img src="./docs/diagrams/cli-terminal.svg" width="50%" alt="Animated terminal demoing the OmniRoute CLI — omniroute providers list, omniroute combo list and omniroute health — cycling over the 86-command top-level surface: providers · oauth · keys · combo · nodes · models · cache · compression · cost · usage · quota · health · resilience · telemetry · logs · audit · mcp · a2a · cloud · memory · skills · eval · tunnel · backup · sync · webhooks · policy · pricing · translator · simulate …"/>
|
||||
|
||||
</div>
|
||||
|
||||
@@ -821,11 +824,11 @@ Expose OmniRoute over **MCP**, **A2A**, a **REST API**, **webhooks** or a **remo
|
||||
<table>
|
||||
<tr><th align="left">Interface</th><th align="left">Endpoint / command</th><th align="left">Use it for</th></tr>
|
||||
<tr><td align="left" nowrap>🧰 <b>MCP (stdio)</b></td><td align="left" nowrap><code>omniroute --mcp</code></td><td align="left">Plug into Claude Desktop, Cursor, any MCP client</td></tr>
|
||||
<tr><td align="left" nowrap>🌊 <b>MCP (HTTP)</b></td><td align="left" nowrap><code>/api/mcp/stream</code></td><td align="left">Remote MCP — <b>110 tools</b>, 33 scopes, full audit trail</td></tr>
|
||||
<tr><td align="left" nowrap>🌊 <b>MCP (HTTP)</b></td><td align="left" nowrap><code>/api/mcp/stream</code></td><td align="left">Remote MCP — <b>110 tools</b>, 33 scopes (enforcement opt-in), full audit trail</td></tr>
|
||||
<tr><td align="left" nowrap>📡 <b>MCP (SSE)</b></td><td align="left" nowrap><code>/api/mcp/sse</code></td><td align="left">Streaming MCP transport</td></tr>
|
||||
<tr><td align="left" nowrap>🤝 <b>A2A</b></td><td align="left" nowrap><code>/.well-known/agent.json</code></td><td align="left">Agent-to-agent, <b>JSON-RPC 2.0</b> + SSE, 6 skills</td></tr>
|
||||
<tr><td align="left" nowrap>🌐 <b>REST API</b></td><td align="left" nowrap><code>/v1/*</code></td><td align="left">OpenAI-compatible — chat, embeddings, images, audio, OCR</td></tr>
|
||||
<tr><td align="left" nowrap>🔔 <b>Webhooks</b></td><td align="left" nowrap><code>/api/webhooks</code></td><td align="left">Push events (usage, quota, errors, routing) to your URL</td></tr>
|
||||
<tr><td align="left" nowrap>🔔 <b>Webhooks</b></td><td align="left" nowrap><code>/api/webhooks</code></td><td align="left">Push request / quota events to Slack, Discord, Telegram or any URL</td></tr>
|
||||
<tr><td align="left" nowrap>🛰️ <b>Remote CLI</b></td><td align="left" nowrap><code>omniroute connect <host></code></td><td align="left">Drive a remote instance with scoped access tokens</td></tr>
|
||||
</table>
|
||||
|
||||
@@ -917,6 +920,8 @@ The 12 engines above shrink what goes **in**. Three more layers shape **how**, *
|
||||
- **🪄 Output Styles** _(output-axis steering)_ — inject deterministic, cache-safe response-shaping instructions; combinable, each at `lite` / `full` / `ultra` intensity. Adding a style is a one-line registry entry:
|
||||
- **Terse prose** — drop filler / articles / hedging; keep technical substance exact.
|
||||
- **Less code** — "lazy senior dev" YAGNI: smallest working change, no unrequested scaffolding.
|
||||
- **Ponytail (lazy senior dev)** — climb the YAGNI ladder, fix the root cause, smallest working diff.
|
||||
- **I have ADHD (action-first)** — next action leads, steps numbered, one concrete next step, no preamble.
|
||||
- **Terse CJK (文言)** — classical-Chinese ultra-terse style (locale-gated to `zh`).
|
||||
- **🎯 Adaptive context-budget** _(the dial)_ — instead of one on/off token threshold, escalate the cheapest, most-lossless engines only as far as needed to **fit the model's context window**. Policy: `reserve-output` (default, model-aware) · `percentage` · `absolute`. Mode: `floor` (guarantee fit) · `replace-autotrigger` (your explicit choice wins) · `off` (legacy threshold).
|
||||
- **🎛️ Where compression is decided** _(precedence, high → low)_ — per-request `x-omniroute-compression` header › routing-combo override › active named profile › adaptive / auto-trigger › panel default › off. The applied plan echoes back in the `X-OmniRoute-Compression: <mode>; source=<source>` response header.
|
||||
@@ -1202,7 +1207,7 @@ Métricas canônicas em 2026-08-24: **1.029 vídeos únicos** · **11.132.922 vi
|
||||
<tr><td nowrap><b>Runtime</b></td><td>Node.js 22.x / 24.x LTS — <code>>=22.22.2 <23 || >=24.0.0 <27</code></td></tr>
|
||||
<tr><td nowrap><b>Language</b></td><td>TypeScript 6.0 — <b>100% TypeScript</b> across <code>src/</code> and <code>open-sse/</code> (zero <code>any</code> in core since v2.0)</td></tr>
|
||||
<tr><td nowrap><b>Framework</b></td><td>Next.js 16 + React 19 + Tailwind CSS 4</td></tr>
|
||||
<tr><td nowrap><b>Database</b></td><td>better-sqlite3 (SQLite, WAL journaling) + LowDB (JSON legacy) — 120 domain modules, 167 migrations</td></tr>
|
||||
<tr><td nowrap><b>Database</b></td><td>better-sqlite3 (SQLite, WAL journaling) + LowDB (JSON legacy) — 122 domain modules, 167 migrations</td></tr>
|
||||
<tr><td nowrap><b>Memory</b></td><td>SQLite FTS5 full-text + int8-quantized vector embeddings, typed decay</td></tr>
|
||||
<tr><td nowrap><b>Schemas</b></td><td>Zod 4 — MCP tool I/O validation + API contracts</td></tr>
|
||||
<tr><td nowrap><b>Protocols</b></td><td>MCP (stdio / HTTP / SSE) + A2A v0.3 (JSON-RPC 2.0 + SSE)</td></tr>
|
||||
@@ -1265,7 +1270,7 @@ Métricas canônicas em 2026-08-24: **1.029 vídeos únicos** · **11.132.922 vi
|
||||
<tr><td nowrap><b><a href="docs/architecture/RESILIENCE_GUIDE.md">Resilience Guide</a></b></td><td>Circuit breakers, cooldowns, queue, anti-thundering herd, TLS spoofing</td></tr>
|
||||
<tr><td nowrap><b><a href="docs/routing/AUTO-COMBO.md">Auto-Combo Engine</a></b></td><td>15-factor scoring, mode packs, self-healing</td></tr>
|
||||
<tr><td nowrap><b><a href="docs/ops/PROXY_GUIDE.md">Proxy Guide</a></b></td><td>3-level proxy system, 1proxy marketplace, registry CRUD</td></tr>
|
||||
<tr><td nowrap><b><a href="docs/reference/FREE_TIERS.md">Free Tiers</a></b></td><td>Consolidated directory: 39 documented recurring pools / 445 cataloged free-tier entries</td></tr>
|
||||
<tr><td nowrap><b><a href="docs/reference/FREE_TIERS.md">Free Tiers</a></b></td><td>Consolidated directory: 38 documented recurring pools / 446 cataloged free-tier entries</td></tr>
|
||||
<tr><td nowrap><b><a href="docs/guides/FEATURES.md">Features Gallery</a></b></td><td>Visual dashboard tour with screenshots</td></tr>
|
||||
<tr><td nowrap><b><a href="docs/architecture/CODEBASE_DOCUMENTATION.md">Codebase Documentation</a></b></td><td>Beginner-friendly codebase walkthrough</td></tr>
|
||||
</table>
|
||||
@@ -1666,7 +1671,7 @@ MIT License - see [LICENSE](LICENSE) for details.
|
||||
|
||||
**[⬆ Back to top](#-omniroute)** · Built with ❤️ for the open-source AI community.
|
||||
|
||||
<sub>OmniRoute v3.8.50 · Node ≥22.22.2 · MIT License · <a href="https://omniroute.online">omniroute.online</a></sub>
|
||||
<sub>OmniRoute v3.8.51 · Node ≥22.22.2 · MIT License · <a href="https://omniroute.online">omniroute.online</a></sub>
|
||||
|
||||
</div>
|
||||
<!-- GitHub Discussions enabled for community Q&A -->
|
||||
|
||||
@@ -829,6 +829,12 @@
|
||||
"src/app/(dashboard)/dashboard/HomePageClient.tsx": {
|
||||
"@typescript-eslint/no-unused-vars": {
|
||||
"count": 2
|
||||
},
|
||||
"react-hooks/exhaustive-deps": {
|
||||
"count": 1
|
||||
},
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 2
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/a2a/page.tsx": {
|
||||
@@ -844,16 +850,62 @@
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/activity/ActivityFeedClient.tsx": {
|
||||
"react-hooks/purity": {
|
||||
"count": 1
|
||||
},
|
||||
"react-hooks/refs": {
|
||||
"count": 1
|
||||
},
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/analytics/CacheHealthTab.tsx": {
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/analytics/ComboHealthTab.tsx": {
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/analytics/CompressionAnalyticsTab.tsx": {
|
||||
"@typescript-eslint/no-unused-vars": {
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/analytics/ProviderUtilizationTab.tsx": {
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/analytics/RouteExplainabilityTab.tsx": {
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 2
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/api-manager/ApiManagerPageClient.tsx": {
|
||||
"@typescript-eslint/no-unused-vars": {
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/audit/A2aAuditTab.tsx": {
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/audit/ComplianceTab.tsx": {
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/audit/McpAuditTab.tsx": {
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 2
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/batch/components/wizard/CostEstimateStep.tsx": {
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 1
|
||||
@@ -874,6 +926,24 @@
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/cache/components/CacheEntriesTab.tsx": {
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/cache/components/ReasoningCacheTab.tsx": {
|
||||
"react-hooks/purity": {
|
||||
"count": 1
|
||||
},
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/cache/page.tsx": {
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/cli-agents/CliAgentsPageClient.tsx": {
|
||||
"no-restricted-syntax": {
|
||||
"count": 4
|
||||
@@ -990,6 +1060,31 @@
|
||||
"count": 2
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/memory/components/EditMemoryModal.tsx": {
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/memory/components/QdrantConfigCard.tsx": {
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/memory/components/tabs/MemoriesTab.tsx": {
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/memory/hooks/useEngineStatus.ts": {
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/memory/hooks/useMemorySettings.ts": {
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/onboarding/page.tsx": {
|
||||
"@typescript-eslint/no-unused-vars": {
|
||||
"count": 1
|
||||
@@ -1090,6 +1185,29 @@
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/radar/RadarCatalogTable.tsx": {
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/radar/intel/page.tsx": {
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/radar/page.tsx": {
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 2
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/radar/setup/page.tsx": {
|
||||
"react-hooks/preserve-manual-memoization": {
|
||||
"count": 1
|
||||
},
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/relay/RelayProxyClient.tsx": {
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 1
|
||||
@@ -1258,6 +1376,11 @@
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/usage/components/EvalsTab.tsx": {
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 2
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/usage/components/ProviderLimits/ProviderLimitCard.tsx": {
|
||||
"@typescript-eslint/no-unused-vars": {
|
||||
"count": 3
|
||||
@@ -1268,6 +1391,11 @@
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/usage/components/ProviderLimits/useCodexResetCreditRedemption.ts": {
|
||||
"react-hooks/immutability": {
|
||||
"count": 2
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/usage/components/ProviderLimits/utils.tsx": {
|
||||
"@typescript-eslint/no-unused-vars": {
|
||||
"count": 1
|
||||
@@ -1276,11 +1404,17 @@
|
||||
"src/app/(dashboard)/dashboard/usage/components/RateLimitStatus.tsx": {
|
||||
"@typescript-eslint/no-unused-vars": {
|
||||
"count": 1
|
||||
},
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/usage/components/SessionsTab.tsx": {
|
||||
"@typescript-eslint/no-unused-vars": {
|
||||
"count": 1
|
||||
},
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/dashboard/webhooks/WebhooksPageClient.tsx": {
|
||||
@@ -1303,6 +1437,17 @@
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/(dashboard)/home/ProviderQuotaWidget.tsx": {
|
||||
"react-hooks/purity": {
|
||||
"count": 1
|
||||
},
|
||||
"react-hooks/refs": {
|
||||
"count": 1
|
||||
},
|
||||
"react-hooks/set-state-in-effect": {
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/app/api/assess/route.ts": {
|
||||
"@typescript-eslint/no-unused-vars": {
|
||||
"count": 1
|
||||
@@ -2153,6 +2298,9 @@
|
||||
"src/shared/components/RequestLoggerV2.tsx": {
|
||||
"@typescript-eslint/no-unused-vars": {
|
||||
"count": 3
|
||||
},
|
||||
"react-hooks/exhaustive-deps": {
|
||||
"count": 6
|
||||
}
|
||||
},
|
||||
"src/shared/components/RequestTimeline.tsx": {
|
||||
|
||||
@@ -202,7 +202,7 @@ Mermaid sources and exported SVG/PNG diagrams referenced from the docs above. Se
|
||||
|
||||
## i18n/
|
||||
|
||||
Translated mirrors of the documentation in 43 locales. See [i18n/README.md](i18n/README.md) for the supported language list.
|
||||
Translated mirrors of the documentation in 42 locales (plus the English originals — 43 languages in total). See [i18n/README.md](i18n/README.md) for the supported language list.
|
||||
|
||||
## screenshots/
|
||||
|
||||
|
||||
@@ -17,27 +17,27 @@ It provides a single OpenAI-compatible endpoint (`/v1/*`) and routes traffic acr
|
||||
|
||||
Core capabilities:
|
||||
|
||||
- OpenAI-compatible API surface for CLI/tools (351 providers, 104 executors)
|
||||
- OpenAI-compatible API surface for CLI/tools (352 providers, 104 executors)
|
||||
- Request/response translation across provider formats
|
||||
- Model combo fallback (multi-model sequence)
|
||||
- Structured combo steps (`provider + model + connection`) with runtime ordering by `compositeTiers`
|
||||
- Account-level fallback (multi-account per provider)
|
||||
- Quota preflight and quota-aware P2C account selection in the main chat path
|
||||
- OAuth + API-key provider connection management (22 OAuth provider modules)
|
||||
- Embedding generation via `/v1/embeddings` (6 providers, 9 models)
|
||||
- Embedding generation via `/v1/embeddings` (18 providers)
|
||||
- Image generation via `/v1/images/generations` (10+ providers, 20+ models)
|
||||
- Audio transcription via `/v1/audio/transcriptions` (7 providers)
|
||||
- Audio transcription via `/v1/audio/transcriptions` (18 providers)
|
||||
- Text-to-speech via `/v1/audio/speech` (24 built-in providers)
|
||||
- Video generation via `/v1/videos/generations` (ComfyUI + SD WebUI)
|
||||
- Music generation via `/v1/music/generations` (ComfyUI)
|
||||
- Web search via `/v1/search` (5 providers)
|
||||
- Web search via `/v1/search` (20 providers)
|
||||
- Moderations via `/v1/moderations`
|
||||
- Reranking via `/v1/rerank`
|
||||
- Think tag parsing (`<think>...</think>`) for reasoning models
|
||||
- Response sanitization for strict OpenAI SDK compatibility
|
||||
- Role normalization (developer→system, system→user) for cross-provider compatibility
|
||||
- Structured output conversion (json_schema → Gemini responseSchema)
|
||||
- Local persistence for providers, keys, aliases, combos, settings, pricing (26 DB modules)
|
||||
- Local persistence for providers, keys, aliases, combos, settings, pricing (122 DB modules)
|
||||
- Usage/cost tracking and request logging
|
||||
- Optional cloud sync for multi-device/state sync
|
||||
- IP allowlist/blocklist for API access control
|
||||
@@ -58,7 +58,7 @@ Core capabilities:
|
||||
- Compliance audit logging with opt-out per API key
|
||||
- Eval framework for LLM quality assurance
|
||||
- Health dashboard with real-time provider circuit breaker status
|
||||
- MCP Server (87 tools) with 3 transports (stdio/SSE/Streamable HTTP)
|
||||
- MCP Server (110 tools) with 3 transports (stdio/SSE/Streamable HTTP)
|
||||
- A2A Server (JSON-RPC 2.0 + SSE) with skills and task lifecycle
|
||||
- Memory system (extraction, injection, retrieval, summarization)
|
||||
- Skills system (registry, executor, sandbox, built-in skills)
|
||||
@@ -330,14 +330,14 @@ OAuth provider modules (22 individual files under `src/lib/oauth/providers/`):
|
||||
## 5) Embedded Services (v3.8.4)
|
||||
|
||||
OmniRoute can install, supervise, and route to locally-running AI tool processes
|
||||
called **embedded services**. Two are shipped in v3.8.4: 9Router and CLIProxyAPI.
|
||||
called **embedded services**. Five are shipped: 9Router, CLIProxyAPI, Bifrost, Mux and Dario.
|
||||
|
||||
Architecture layers:
|
||||
|
||||
- **UI** (`/dashboard/providers/services`) — two-tab page with lifecycle controls,
|
||||
live log streaming, API key management, and (for 9Router) embedded native UI via
|
||||
an internal reverse proxy.
|
||||
- **API** (`/api/services/{name}/*`) — 8 endpoints for 9Router, 7 for CLIProxyAPI,
|
||||
- **API** (`/api/services/{name}/*`) — 11 endpoints for 9Router, 10 for CLIProxyAPI, 8 each for Bifrost / Mux / Dario,
|
||||
all classified **LOCAL_ONLY** (hard rule #17). A shared `GET /api/services/[name]/logs`
|
||||
SSE endpoint serves both services.
|
||||
- **Supervisor** (`src/lib/services/`) — generic `ServiceSupervisor` class wraps
|
||||
@@ -879,8 +879,8 @@ flowchart LR
|
||||
|
||||
- `open-sse/translator/index.ts`: translator registry and orchestration
|
||||
- Request translators: `open-sse/translator/request/*` (9 modules — `antigravity-to-openai`, `claude-to-gemini`, `claude-to-openai`, `gemini-to-openai`, `openai-responses`, `openai-to-claude`, `openai-to-cursor`, `openai-to-gemini`, `openai-to-kiro`)
|
||||
- Response translators: `open-sse/translator/response/*` (8 modules — `claude-to-openai`, `cursor-to-openai`, `gemini-to-claude`, `gemini-to-openai`, `kiro-to-openai`, `openai-responses`, `openai-to-antigravity`, `openai-to-claude`)
|
||||
- Helpers: `open-sse/translator/helpers/*` (8 modules — `claudeHelper`, `geminiHelper`, `geminiToolsSanitizer`, `maxTokensHelper`, `openaiHelper`, `responsesApiHelper`, `schemaCoercion`, `toolCallHelper`)
|
||||
- Response translators: `open-sse/translator/response/*` (11 modules — `claude-to-openai`, `cursor-to-openai`, `gemini-to-claude`, `gemini-to-openai`, `kiro-to-openai`, `openai-responses`, `openai-to-antigravity`, `openai-to-claude`, `openai-to-gemini`, `openai-to-gemini-sse`, `responsesToolItem`)
|
||||
- Helpers: `open-sse/translator/helpers/*` (12 modules — `claudeHelper`, `geminiHelper`, `geminiToolsSanitizer`, `jsonUtil`, `markdownBoundary`, `maxTokensHelper`, `openaiHelper`, `responsesApiHelper`, `schemaCoercion`, `strictSystemHoist`, `toolCallHelper`, `toolCallShim`)
|
||||
- Format constants: `open-sse/translator/formats.ts`
|
||||
- Bootstrap and registry: `open-sse/translator/bootstrap.ts`, `open-sse/translator/registry.ts`
|
||||
- Image-format helpers: `open-sse/translator/image/`
|
||||
|
||||
@@ -6,7 +6,7 @@ lastUpdated: 2026-06-28
|
||||
|
||||
# OmniRoute Codebase Documentation
|
||||
|
||||
> **Version:** v3.8.0
|
||||
> **Version:** v3.8.51
|
||||
> **Last updated:** 2026-06-28
|
||||
> **Audience:** Engineers contributing to OmniRoute or building integrations on top of it.
|
||||
>
|
||||
@@ -64,7 +64,7 @@ OmniRoute/
|
||||
├── _ideia/, _references/, _mono_repo/, _tasks/ Internal scratch / planning (not shipped)
|
||||
├── CLAUDE.md Repo rules for Claude Code
|
||||
├── AGENTS.md Deeper architecture reference for agents
|
||||
├── package.json v3.8.0, workspace root
|
||||
├── package.json v3.8.51, workspace root
|
||||
└── tsconfig.json Path aliases + core compiler options
|
||||
```
|
||||
|
||||
@@ -219,7 +219,7 @@ src/app/api/services/
|
||||
Corresponding dashboard UI:
|
||||
`src/app/(dashboard)/dashboard/providers/services/` — two-tab page (CLIProxyAPI + 9Router).
|
||||
Reverse proxy for 9Router embedded UI:
|
||||
`src/app/(dashboard)/dashboard/providers/services/[name]/embed/[...path]/route.ts`
|
||||
`src/app/(dashboard)/dashboard/providers/services/[name]/embed/[[...path]]/route.ts`
|
||||
|
||||
Deep-dive: `docs/frameworks/EMBEDDED-SERVICES.md`
|
||||
|
||||
@@ -349,10 +349,10 @@ Domain modules (each owns one or more tables): `apiKeys.ts`, `backup.ts`,
|
||||
`syncTokens.ts`, `tierConfig.ts`, `upstreamProxy.ts`, `versionManager.ts`,
|
||||
`webhooks.ts`.
|
||||
|
||||
`migrations/` holds 55 versioned `.sql` files (idempotent, transactional) and is
|
||||
`migrations/` holds 167 versioned `.sql` files (idempotent, transactional) and is
|
||||
executed by `migrationRunner.ts` at boot.
|
||||
|
||||
Tables created across the migrations (52 total):
|
||||
Tables created across the migrations (123 total):
|
||||
|
||||
`a`, `account_key_limits`, `api_keys`, `batches`, `call_logs`,
|
||||
`combo_adaptation_state`, `combos`, `command_code_auth_sessions`,
|
||||
@@ -450,12 +450,12 @@ open-sse/
|
||||
├── types.d.ts
|
||||
├── config/ Provider registries, header profiles, identity, …
|
||||
├── handlers/ Request handlers (chat, embeddings, audio, image, …)
|
||||
├── executors/ 108 provider-specific HTTP executors
|
||||
├── executors/ 104 provider-specific HTTP executors
|
||||
├── translator/ Format conversion (OpenAI ↔ Claude ↔ Gemini ↔ Cursor ↔ Kiro)
|
||||
├── transformer/ Responses API ↔ Chat Completions stream transformer
|
||||
├── services/ 80+ service modules (combos, fallback, quotas, identity, …)
|
||||
├── utils/ Streaming helpers, TLS client, AWS SigV4, proxy fetch, …
|
||||
└── mcp-server/ MCP server (3 transports, 31 scopes, 105 tools)
|
||||
└── mcp-server/ MCP server (3 transports, 33 scopes, 110 tools)
|
||||
```
|
||||
|
||||
### 4.1 `open-sse/handlers/`
|
||||
@@ -489,7 +489,7 @@ open-sse/
|
||||
(shared identity helper) and `index.ts` (registry).
|
||||
|
||||
> Note: providers not listed here are served by `default.ts` using the generic
|
||||
> OpenAI-compatible executor. The full provider catalog (351 providers) lives in
|
||||
> OpenAI-compatible executor. The full provider catalog (352 providers) lives in
|
||||
> `src/shared/constants/providers.ts`.
|
||||
|
||||
### 4.3 `open-sse/translator/`
|
||||
@@ -538,11 +538,11 @@ Highlights (full list under `open-sse/services/`):
|
||||
|
||||
### 4.6 `open-sse/mcp-server/`
|
||||
|
||||
- **31 registered tools** wired in `server.ts` (12 scoped under `schemas/tools.ts`,
|
||||
5 compression tools, 3 memory tools, 4 skills tools, plus advanced tools added
|
||||
through `advancedTools.ts`).
|
||||
- **110 unique tools** wired in `server.ts` (45 canonical in `schemas/tools.ts` +
|
||||
memory, skills, GitHub-skills, pool, gamification, plugin, Notion, Obsidian,
|
||||
local-corpus and compression modules — union counted by `countUniqueMcpTools`).
|
||||
- **3 transports**: stdio, HTTP Streamable, SSE.
|
||||
- **31 scopes** declared in `src/shared/constants/mcpScopes.ts`.
|
||||
- **33 scopes** enforced at runtime — base list in `src/shared/constants/mcpScopes.ts`, full set is the union of the scopes declared by each tool module.
|
||||
- Audit table: `mcp_tool_audit` (populated by `audit.ts`).
|
||||
- Files: `server.ts`, `index.ts`, `httpTransport.ts`, `audit.ts`, `scopeEnforcement.ts`,
|
||||
`runtimeHeartbeat.ts`, `descriptionCompressor.ts`, `schemas/{tools, a2a, audit, index}.ts`,
|
||||
@@ -585,7 +585,7 @@ electron/
|
||||
├── main.js Electron main process
|
||||
├── preload.js Preload bridge (contextIsolation enabled)
|
||||
├── types.d.ts
|
||||
├── package.json electron-builder config, version 3.8.0
|
||||
├── package.json electron-builder config, version 3.8.51
|
||||
├── README.md
|
||||
├── assets/ Build resources (icons, entitlements, …)
|
||||
├── node_modules/ Dedicated node_modules (better-sqlite3, electron-updater)
|
||||
@@ -637,7 +637,7 @@ Two binaries are exposed in `package.json` → `bin`:
|
||||
| `tests/unit/` | Unit tests via Node native test runner (1821 files, plus `api/`, `auth/`, `authz/` subdirs) |
|
||||
| `tests/integration/` | Cross-module + DB-state tests |
|
||||
| `tests/e2e/` | Playwright UI tests |
|
||||
| `tests/protocols-e2e/` | MCP/A2A protocol e2e |
|
||||
| `tests/e2e/protocol-clients.test.ts` | MCP/A2A protocol e2e |
|
||||
| `tests/translator/` | Translator-specific tests |
|
||||
| `tests/security/` | Security regressions |
|
||||
| `tests/load/` | Load / stress tests |
|
||||
|
||||
@@ -9,13 +9,13 @@ It describes each gate, what it validates, which CI job it runs in, whether it u
|
||||
a ratchet baseline or a pass/fail policy, and whether it blocks the build or is advisory.
|
||||
|
||||
For a short summary and the allowlist policy, see the "Quality Gates & Ratchets" section
|
||||
in `CLAUDE.md`. For the critical assessment, maturity classification, and tool-agnostic
|
||||
in `AGENTS.md`. For the critical assessment, maturity classification, and tool-agnostic
|
||||
replication plan of the same system, see the
|
||||
[Quality Gate Playbook](../ops/QUALITY_GATE_PLAYBOOK.md).
|
||||
|
||||
---
|
||||
|
||||
## Gate Inventory (~50 scripts)
|
||||
## Gate Inventory (~90 scripts)
|
||||
|
||||
Scripts live under `scripts/check/` (policy gates) and `scripts/quality/` (ratchet engine).
|
||||
The CI source of truth is `.github/workflows/ci.yml`.
|
||||
@@ -57,47 +57,50 @@ assertion weakening and other masking remain owned by the independently blocking
|
||||
|
||||
Runs on every PR to `main`. Blocks merge on failure.
|
||||
|
||||
| Script (`npm run ...`) | Validates | Blocking |
|
||||
| ------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------- |
|
||||
| `check:node-runtime` | Node.js version is within the supported range | Yes |
|
||||
| `check:cycles` | Circular imports — all `src/` + `open-sse/` modules | Yes |
|
||||
| `check:route-validation:t06` | Zod schemas present on all routes (Tier 6 policy) | Yes |
|
||||
| `check:any-budget:t11` | `@ts-expect-error // any` count does not exceed budget (Tier 11 catraca) | Yes |
|
||||
| `check:provider-consistency` | Every provider in `providers.ts` has a matching entry in `providerRegistry.ts` (and vice-versa, within the allowlist) | Yes |
|
||||
| `check:model-lifecycle` | The two hand-maintained routing tables do not point at retired models (#11503): `FITNESS_TABLE` (`taskFitness.ts`) scores no routable retired id, every `BUILT_IN_ALIASES` target is a live catalog model, and every retired id the catalog still routes is either forwarded or listed in `allowedRetiredInCatalog`. Offline — compares against the vendor snapshot `config/quality/model-lifecycle.json`, refreshed by hand with `npm run quality:refresh-model-lifecycle` (network; not wired into CI). `allowedRetiredInCatalog` is a burn-down ratchet: add an entry only with a tracking issue. | Yes |
|
||||
| `check:fetch-targets` | Every `fetch("/api/...")` in client-side `src/` resolves to a real `route.ts` | Yes |
|
||||
| `check:deps` | All `npm install`-able deps across every `package.json` in the repo are in `dependency-allowlist.json`; new unpinned or slopsquatted packages flagged | Yes |
|
||||
| `audit:deps` | `npm audit` (root + electron) — no high/critical advisories (overlaps osv `check:vuln-ratchet`; see Rationalization Backlog) | Yes |
|
||||
| `check:lockfile` | `package-lock.json` integrity — https registry, integrity hashes, no host overrides | Yes |
|
||||
| `check:licenses` | SPDX license allowlist for production dependencies | Yes |
|
||||
| `check:tracked-artifacts` | No build artifacts / committed `node_modules` symlinks (also runs in husky pre-commit; pre-push is intentionally light — #6716) | Yes |
|
||||
| `check:file-size` | No source file exceeds the per-extension cap (ratchet: frozen large files in `frozen` list) | Yes |
|
||||
| `check:error-helper` | Error responses in executors/handlers use `buildErrorBody()` / `sanitizeErrorMessage()` (Hard Rule #12) | Yes |
|
||||
| `check:migration-numbering` | Migration SQL files are sequentially numbered, no gaps or duplicates | Yes |
|
||||
| `check:public-creds` | No literal OAuth `client_id`/`client_secret` or Firebase Web keys outside `publicCreds.ts` (Hard Rule #11) | Yes |
|
||||
| `check:db-rules` | No raw SQL outside `src/lib/db/` modules; no barrel-imports from `localDb.ts` (Hard Rules #2/#5) | Yes |
|
||||
| `check:known-symbols` | Provider executors, routing strategies, and translators registered in their dispatch tables match the files on disk — no orphaned or undeclared symbols | Yes |
|
||||
| `check:route-guard-membership` | Every route that spawns a child process is classified by `isLocalOnlyPath()` (Hard Rules #15/#17) | Yes |
|
||||
| `check:test-discovery` | Every `*.test.ts` / `*.spec.ts` file in the repo is collected by at least one test runner (ratchet: orphan list in `test-discovery-baseline.json` can only shrink) | Yes |
|
||||
| `check:docs-sync` | CHANGELOG version, OpenAPI version, and `llm.txt` are in sync | Yes |
|
||||
| `typecheck:core` | TypeScript compilation without errors (advisory warnings only) | Yes |
|
||||
| `typecheck:noimplicit:core` | Strict `noImplicitAny` — forward-looking; many pre-existing call sites still need annotations | **Advisory** (`continue-on-error: true`) |
|
||||
| `check:dashboard-typecheck` | `tsc` scoped to `src/app/(dashboard)/**` (#7033) — `typecheck:core`'s curated 27-file allowlist does not include any dashboard TSX, and `next build` never type-checks it either (`next.config.mjs` sets `ignoreBuildErrors: true`), so orphaned-identifier regressions there (#6625/#6909) were invisible to CI. Diffs against a frozen per-file/per-TS-code count baseline (`config/quality/dashboard-typecheck-baseline.json`, same stale-enforcement pattern as `check:known-symbols`) — only NEW errors beyond the baselined count fail the gate; ratchet down with `--update` when a pre-existing error is fixed. | Yes |
|
||||
| Script (`npm run ...`) | Validates | Blocking |
|
||||
| --------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------- |
|
||||
| `check:node-runtime` | Node.js version is within the supported range | Yes |
|
||||
| `check:cycles` | Circular imports — all `src/` + `open-sse/` modules | Yes |
|
||||
| `check:route-validation:t06` | Zod schemas present on all routes (Tier 6 policy) | Yes |
|
||||
| `check:any-budget:t11` | `@ts-expect-error // any` count does not exceed budget (Tier 11 catraca) | Yes |
|
||||
| `check:provider-consistency` | Every provider in `providers.ts` has a matching entry in `providerRegistry.ts` (and vice-versa, within the allowlist) | Yes |
|
||||
| `check:model-lifecycle` | The two hand-maintained routing tables do not point at retired models (#11503): `FITNESS_TABLE` (`taskFitness.ts`) scores no routable retired id, every `BUILT_IN_ALIASES` target is a live catalog model, and every retired id the catalog still routes is either forwarded or listed in `allowedRetiredInCatalog`. Offline — compares against the vendor snapshot `config/quality/model-lifecycle.json`, refreshed by hand with `npm run quality:refresh-model-lifecycle` (network; not wired into CI). `allowedRetiredInCatalog` is a burn-down ratchet: add an entry only with a tracking issue. | Yes |
|
||||
| `check:fetch-targets` | Every `fetch("/api/...")` in client-side `src/` resolves to a real `route.ts` | Yes |
|
||||
| `check:deps` | All `npm install`-able deps across every `package.json` in the repo are in `dependency-allowlist.json`; new unpinned or slopsquatted packages flagged | Yes |
|
||||
| `audit:deps` | `npm audit` (root + electron) — no high/critical advisories (overlaps osv `check:vuln-ratchet`; see Rationalization Backlog) | Yes |
|
||||
| `check:lockfile` | `package-lock.json` integrity — https registry, integrity hashes, no host overrides | Yes |
|
||||
| `check:licenses` | SPDX license allowlist for production dependencies | Yes |
|
||||
| `check:tracked-artifacts` | No build artifacts / committed `node_modules` symlinks (also runs in husky pre-commit; pre-push is intentionally light — #6716) | Yes |
|
||||
| `check:file-size` | No source file exceeds the per-extension cap (ratchet: frozen large files in `frozen` list) | Yes |
|
||||
| `check:error-helper` | Error responses in executors/handlers use `buildErrorBody()` / `sanitizeErrorMessage()` (Hard Rule #12) | Yes |
|
||||
| `check:migration-numbering` | Migration SQL files are sequentially numbered, no gaps or duplicates | Yes |
|
||||
| `check:public-creds` | No literal OAuth `client_id`/`client_secret` or Firebase Web keys outside `publicCreds.ts` (Hard Rule #11) | Yes |
|
||||
| `check:db-rules` | No raw SQL outside `src/lib/db/` modules; no barrel-imports from `localDb.ts` (Hard Rules #2/#5) | Yes |
|
||||
| `check:known-symbols` | Provider executors, routing strategies, and translators registered in their dispatch tables match the files on disk — no orphaned or undeclared symbols | Yes |
|
||||
| `check:route-guard-membership` | Every route that spawns a child process is classified by `isLocalOnlyPath()` (Hard Rules #15/#17) | Yes |
|
||||
| `check:test-discovery` | Every `*.test.ts` / `*.spec.ts` file in the repo is collected by at least one test runner (ratchet: orphan list in `test-discovery-baseline.json` can only shrink) | Yes |
|
||||
| `check:agent-skills-sync` | Generated agent-skills artifacts match their source catalog (no drift) |
|
||||
| `check:provider-asset-provenance` | Provider logos/assets carry a recorded provenance entry |
|
||||
| `lint:json` | JSON config files parse and satisfy the repo lint rules |
|
||||
| `typecheck:core` | TypeScript compilation without errors (advisory warnings only) | Yes |
|
||||
| `typecheck:noimplicit:core` | Strict `noImplicitAny` — forward-looking; many pre-existing call sites still need annotations | **Advisory** (`continue-on-error: true`) |
|
||||
| `check:dashboard-typecheck` | `tsc` scoped to `src/app/(dashboard)/**` (#7033) — `typecheck:core`'s curated 27-file allowlist does not include any dashboard TSX, and `next build` never type-checks it either (`next.config.mjs` sets `ignoreBuildErrors: true`), so orphaned-identifier regressions there (#6625/#6909) were invisible to CI. Diffs against a frozen per-file/per-TS-code count baseline (`config/quality/dashboard-typecheck-baseline.json`, same stale-enforcement pattern as `check:known-symbols`) — only NEW errors beyond the baselined count fail the gate; ratchet down with `--update` when a pre-existing error is fixed. | Yes |
|
||||
|
||||
### Job: `quality-gate`
|
||||
|
||||
Runs after `test-coverage`. Blocks merge on failure.
|
||||
|
||||
| Script | Validates | Blocking |
|
||||
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | ------------------------- |
|
||||
| `quality:collect` | Emits `quality-metrics.json` (ESLint warning count, coverage from merged shard report) | Yes (upstream of ratchet) |
|
||||
| `quality:ratchet` | Each metric in `quality-baseline.json` has not regressed (ESLint warnings ≤ baseline; coverage ≥ baseline) | Yes |
|
||||
| `check:duplication` | Code duplication (jscpd@4) does not exceed baseline in `quality-baseline.json` | Yes |
|
||||
| `check:complexity` | File-level cyclomatic complexity does not exceed the cap (core ESLint `complexity` + `max-lines-per-function`) | Yes |
|
||||
| `check:cognitive-complexity` | Cognitive complexity ratchet (`eslint-plugin-sonarjs`) — separate ESLint pass; mergeable with `check:complexity` (see Backlog) | Yes |
|
||||
| `check:dead-code` | Unused exports / files ratchet (knip) does not regress vs baseline | Yes |
|
||||
| `check:type-coverage` | Percent-typed ratchet (`type-coverage`) does not regress; largely subsumes `typecheck:noimplicit:core` | Yes |
|
||||
| `check:codeql-ratchet` | Open CodeQL alert count does not regress (reads via `gh api`; graceful-skip without token) | Yes |
|
||||
| Script | Validates | Blocking |
|
||||
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------- |
|
||||
| `quality:collect` | Emits `quality-metrics.json` (ESLint warning count, coverage from merged shard report) | Yes (upstream of ratchet) |
|
||||
| `quality:ratchet` | Each metric in `quality-baseline.json` has not regressed (ESLint warnings ≤ baseline; coverage ≥ baseline) | Yes |
|
||||
| `check:duplication` | Code duplication (jscpd@4) does not exceed baseline in `quality-baseline.json` | Yes |
|
||||
| `check:complexity` | File-level cyclomatic complexity does not exceed the cap (core ESLint `complexity` + `max-lines-per-function`) | Yes |
|
||||
| `check:cognitive-complexity` | Cognitive complexity ratchet (`eslint-plugin-sonarjs`) — separate ESLint pass; CI runs both merged as the single `check:complexity-ratchets` step | Yes |
|
||||
| `check:dead-code` | Unused exports / files ratchet (knip) does not regress vs baseline | Yes |
|
||||
| `check:compression-budget` | Compression benchmark budget — per-engine token-savings floors must not regress | Yes |
|
||||
| `check:type-coverage` | Percent-typed ratchet (`type-coverage`) does not regress; largely subsumes `typecheck:noimplicit:core` | Yes |
|
||||
| `check:codeql-ratchet` | Open CodeQL alert count does not regress (reads via `gh api`; graceful-skip without token) | Yes |
|
||||
|
||||
### Job: `quality-extended`
|
||||
|
||||
@@ -208,7 +211,7 @@ Runs after `build`. Blocks merge on failure.
|
||||
|
||||
| Suite | Validates | Blocking |
|
||||
| ---------------- | -------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
|
||||
| `test:vitest` | MCP server (109 tools), autoCombo, cache — vitest runner | Yes |
|
||||
| `test:vitest` | MCP server (110 tools), autoCombo, cache — vitest runner | Yes |
|
||||
| `test:vitest:ui` | UI component tests — vitest runner | **Blocking** — pre-existing failures are explicitly excluded in `vitest.config.ts`; new failures fail the job |
|
||||
|
||||
### Nightly workflows (scheduled, advisory)
|
||||
@@ -221,6 +224,8 @@ These run on a cron schedule (and `workflow_dispatch`), never on PRs. All are ad
|
||||
| `nightly-resilience` | heap-growth gate, chaos fault-injection, k6 load/soak | **Advisory** |
|
||||
| `nightly-llm-security` | promptfoo injection guard (block mode) + garak probes (skipped without a provider secret) | **Advisory** |
|
||||
| `nightly-schemathesis` | OpenAPI contract fuzzing (schemathesis) against a live OmniRoute using `docs/openapi.yaml` — surfaces spec violations / unhandled 500s (Fase 8 B.4) | **Advisory** |
|
||||
| `nightly-mutation` | Stryker mutation-testing score over the fast unit lane — surviving mutants surface weak asserts | **Advisory** |
|
||||
| `nightly-compat` | Node engine compatibility matrix across the supported `engines.node` ranges | **Advisory** |
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -21,7 +21,7 @@ OmniRoute/
|
||||
├── bin/ # CLI entry point and command handlers
|
||||
├── scripts/ # Build, check, sync, and one-off scripts
|
||||
├── docs/ # Public documentation (you are here)
|
||||
├── tests/ # All test suites (unit, integration, e2e, protocols-e2e)
|
||||
├── tests/ # All test suites (unit, integration, e2e, protocol clients)
|
||||
├── public/ # Next.js static assets, PWA manifest, service worker, icons
|
||||
├── config/ # Static config + quality-gate state (i18n, payloadRules, quality/)
|
||||
├── images/ # Marketing / README image assets
|
||||
@@ -117,11 +117,11 @@ OmniRoute/
|
||||
```
|
||||
src/
|
||||
├── app/ # App Router (pages + API routes + status pages + landing)
|
||||
├── lib/ # Core libraries / domain modules (~50 subdirs + ~30 top-level files)
|
||||
├── lib/ # Core libraries / domain modules (80 subdirs + ~70 top-level files)
|
||||
├── domain/ # Pure domain logic (policy engine, fallback, cost, lockout, comboResolver, assessment)
|
||||
├── server/ # Server-only modules (authz pipeline, cors, auth middleware) — cannot import from client
|
||||
├── shared/ # Shared between server and client where safe (constants, types, validation, contracts, utils)
|
||||
├── i18n/ # next-intl config + per-locale message JSON (30+ locales)
|
||||
├── i18n/ # next-intl config + per-locale message JSON (43 locales)
|
||||
├── middleware/ # Next.js middleware (request enrichment, locale detection)
|
||||
├── mitm/ # MITM proxy core: cert gen/install, handlers, targets, inspector, masks, passthrough
|
||||
│ ├── handlers/ # 9 IDE-agent handler classes extending MitmHandlerBase (antigravity, kiro, copilot, codex, cursor, zed, claudeCode, openCode, trae)
|
||||
@@ -148,9 +148,8 @@ src/
|
||||
| `app/api/tools/traffic-inspector/` | Traffic Inspector REST + WS API — 16+ routes (requests, sessions, hosts, capture-modes, export, ws). LOCAL_ONLY + SPAWN_CAPABLE. See `docs/frameworks/TRAFFIC_INSPECTOR.md §8`. |
|
||||
| `app/a2a/` | A2A JSON-RPC 2.0 entry point (`POST /a2a`) |
|
||||
| `app/.well-known/agent.json/` | A2A Agent Card (discovery) |
|
||||
| `app/(dashboard)/dashboard/` | Dashboard UI pages (~35 pages: providers, combos, settings, memory, skills, webhooks, evals, audit, batch, cache, costs, health, system, activity, etc.) |
|
||||
| `app/(dashboard)/dashboard/` | Dashboard UI pages (50+ sections, ~118 page.tsx files: providers, combos, settings, memory, skills, webhooks, evals, audit, batch, cache, costs, health, system, activity, etc.) |
|
||||
| `app/(dashboard)/dashboard/search-tools/` | Search Tools Studio UI (3 tabs: Search/Scrape/Compare + SearchConceptCard + ProviderCatalog) — see `docs/frameworks/SEARCH_TOOLS_STUDIO.md` |
|
||||
| `app/(dashboard)/dashboard/` | Dashboard UI pages (~30 pages: providers, combos, settings, memory, skills, webhooks, evals, audit, batch, cache, costs, health, system, etc.) |
|
||||
| `app/(dashboard)/dashboard/memory/` | Memory Studio (plan 21): `page.tsx` (3-tab shell), `components/` (MemoryConceptCard, MemoryEngineStatus, EmbeddingSourceSelector, EditMemoryModal, RetrievePreview, QdrantConfigCard, RerankConfigCard), `components/tabs/` (MemoriesTab, PlaygroundTab, EngineTab), `hooks/` (useEngineStatus, useMemorySettings) |
|
||||
| `app/(dashboard)/dashboard/tools/agent-bridge/` | AgentBridge dashboard page — server card, 9 agent cards, setup wizard, model mapping, bypass list. i18n PT-BR + EN. See `docs/frameworks/AGENTBRIDGE.md`. |
|
||||
| `app/(dashboard)/dashboard/tools/traffic-inspector/` | Traffic Inspector dashboard page — DevTools split, 7 detail tabs, 4 capture mode toggles, session recorder, context colorization. i18n PT-BR + EN. See `docs/frameworks/TRAFFIC_INSPECTOR.md`. |
|
||||
@@ -181,7 +180,7 @@ src/
|
||||
| `compliance/` | Audit log + provider audit — see `docs/security/COMPLIANCE.md` |
|
||||
| `compression/` | Compression engine glue (engines live in `open-sse/services/compression/`) |
|
||||
| `config/` | Runtime config helpers |
|
||||
| `db/` | 95+ domain DB modules + 148 migrations (always go through here for SQLite) |
|
||||
| `db/` | 120+ domain DB modules + 167 migrations (always go through here for SQLite) |
|
||||
| `quota/` | Quota Sharing Engine: `dimensions.ts` (types/Zod), `types.ts` (QuotaStore interface), `sqliteQuotaStore.ts`, `redisQuotaStore.ts`, `storeFactory.ts`, `fairShare.ts`, `burnRate.ts`, `planResolver.ts`, `planRegistry.ts`, `saturationSignals.ts`, `enforce.ts`, `spendRecorder.ts` — see `docs/routing/QUOTA_SHARE.md` |
|
||||
| `radar/` | Radar free-model catalog client: `feedSchema.ts`, `pinnedKeys.ts`, `verify.ts`, `sync.ts`, `applyFeed.ts`, `index.ts` (`getRadarCatalog()`) — see `docs/frameworks/RADAR.md` |
|
||||
| `display/` | UI formatting helpers (cost, latency, etc.) |
|
||||
@@ -208,7 +207,7 @@ src/
|
||||
| `cacheLayer.ts`, `idempotencyLayer.ts` | Request caching + idempotency |
|
||||
| (~30 more top-level files) | Specialized helpers (logEnv, modelsDevSync, piiSanitizer, etc.) |
|
||||
|
||||
### `src/db/` — Database (117 modules + 148 migrations)
|
||||
### `src/lib/db/` — Database (122 modules + 167 migrations)
|
||||
|
||||
| Subdir | Purpose |
|
||||
| ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
@@ -220,16 +219,16 @@ src/
|
||||
|
||||
### `src/domain/`
|
||||
|
||||
| Module | Purpose |
|
||||
| ---------------------- | ----------------------------------------------------------------------- |
|
||||
| `policy.ts` | Policy engine |
|
||||
| `fallbackPolicy.ts` | Fallback decision tree |
|
||||
| `costRules.ts` | Cost calculation rules |
|
||||
| `lockoutPolicy.ts` | Model/connection lockout policy |
|
||||
| `tagRouter.ts` | Tag-based routing |
|
||||
| `comboResolver.ts` | Combo resolution (used by combo engine) |
|
||||
| `modelAvailability.ts` | Per-model availability check |
|
||||
| `assessment/` | Model assessment (Phase 1 of RFC-AUTO-ASSESSMENT — see `docs/archive/`) |
|
||||
| Module | Purpose |
|
||||
| ---------------------- | -------------------------------------------------- |
|
||||
| `policy.ts` | Policy engine |
|
||||
| `fallbackPolicy.ts` | Fallback decision tree |
|
||||
| `costRules.ts` | Cost calculation rules |
|
||||
| `lockoutPolicy.ts` | Model/connection lockout policy |
|
||||
| `tagRouter.ts` | Tag-based routing |
|
||||
| `comboResolver.ts` | Combo resolution (used by combo engine) |
|
||||
| `modelAvailability.ts` | Per-model availability check |
|
||||
| `assessment/` | Model assessment (Phase 1 of RFC-AUTO-ASSESSMENT ) |
|
||||
|
||||
### `src/server/`
|
||||
|
||||
@@ -243,7 +242,7 @@ src/
|
||||
|
||||
| Module | Purpose |
|
||||
| -------------------------------- | ---------------------------------------------------------------------- |
|
||||
| `constants/providers.ts` | **338 providers** with Zod validation (source of truth) |
|
||||
| `constants/providers.ts` | **352 providers** with Zod validation (source of truth) |
|
||||
| `constants/cliTools.ts` | External CLI tool registry |
|
||||
| `constants/routingStrategies.ts` | **19 routing strategies** with priorities |
|
||||
| `constants/publicApiRoutes.ts` | Routes that require Bearer (vs management) auth |
|
||||
@@ -270,7 +269,7 @@ open-sse/
|
||||
├── translator/ # Format converters (9 request, 9 response, 9 helpers)
|
||||
├── transformer/ # Responses API ↔ Chat Completions (TransformStream)
|
||||
├── services/ # ~80+ service modules (combo, accountFallback, autoCombo, reasoningCache, claude code/chatgpt stealth, modelDeprecation, taskAwareRouter, workflowFSM, etc.)
|
||||
├── mcp-server/ # MCP server (109 tools, 3 transports, 33 scopes)
|
||||
├── mcp-server/ # MCP server (110 tools, 3 transports, 33 scopes)
|
||||
├── config/ # Provider/model registries, header config, model aliases
|
||||
├── utils/ # TLS client, proxy fetch/dispatcher, network helpers
|
||||
├── index.ts # Workspace entry
|
||||
@@ -379,7 +378,7 @@ open-sse/
|
||||
|
||||
---
|
||||
|
||||
## `docs/` — Public Documentation (44 files + 4 subdirs)
|
||||
## `docs/` — Public Documentation (7 root files + 17 subdirs)
|
||||
|
||||
### Top-level guides
|
||||
|
||||
@@ -392,7 +391,7 @@ open-sse/
|
||||
| `API_REFERENCE.md` | API endpoint reference with auth model |
|
||||
| `openapi.yaml` | OpenAPI 3.0 spec (121 paths) |
|
||||
| `SETUP_GUIDE.md` | Install methods (npm, npx, Docker, Electron, Termux, source) |
|
||||
| `ENVIRONMENT.md` | All env vars (~219 used in code, ~810 lines `.env.example`) |
|
||||
| `ENVIRONMENT.md` | All env vars (~800 documented, ~3,050 lines `.env.example`) |
|
||||
| `TROUBLESHOOTING.md` | Common errors + v3.8.0 known issues |
|
||||
| `RELEASE_CHECKLIST.md` | Full release flow (skills, husky, conventional commits, deploy) |
|
||||
| `COVERAGE_PLAN.md` | Coverage goals and current state |
|
||||
@@ -400,13 +399,13 @@ open-sse/
|
||||
| `CLI-TOOLS.md` | External CLI integrations + Internal OmniRoute CLI |
|
||||
| `I18N.md` | i18n architecture, adding a language, 43 locales |
|
||||
| `UNINSTALL.md` | Clean uninstall steps |
|
||||
| `PROVIDER_REFERENCE.md` | **Auto-generated** catalog of 338 providers (regen: `npm run gen:provider-reference`) |
|
||||
| `PROVIDER_REFERENCE.md` | **Auto-generated** catalog of 352 providers (regen: `npm run gen:provider-reference`) |
|
||||
|
||||
### Subsystem deep-dives
|
||||
|
||||
| Doc | Purpose |
|
||||
| -------------------------- | ------------------------------------------------------------------- |
|
||||
| `MCP-SERVER.md` | MCP server: 109 tools, 3 transports, 33 scopes, REST endpoints |
|
||||
| `MCP-SERVER.md` | MCP server: 110 tools, 3 transports, 33 scopes, REST endpoints |
|
||||
| `A2A-SERVER.md` | A2A v0.3: JSON-RPC, 6 skills, REST helpers, agent card |
|
||||
| `AGENT_PROTOCOLS_GUIDE.md` | Unified guide: A2A vs ACP vs Cloud Agents |
|
||||
| `CLOUD_AGENT.md` | Codex Cloud / Devin / Jules orchestration |
|
||||
@@ -450,8 +449,7 @@ open-sse/
|
||||
|
||||
| Subdir | Purpose |
|
||||
| --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `docs/archive/` | Archived/historical docs (e.g., `RFC-AUTO-ASSESSMENT-DRAFT.md` — superseded by EVALS) |
|
||||
| `docs/i18n/` | Localized doc translations (~42 locales) |
|
||||
| `docs/i18n/` | Localized doc translations (42 locales) |
|
||||
| `docs/screenshots/` | Image assets for guides |
|
||||
| `_tasks/superpowers/` | Plans/specs from superpowers (`writing-plans`/`brainstorming`) + research — isolated, separately-versioned repo, gitignored by the main tree. See CLAUDE.md → "Planning & Research Artifacts". |
|
||||
|
||||
@@ -459,13 +457,13 @@ open-sse/
|
||||
|
||||
## `tests/` — Test Suites
|
||||
|
||||
| Subdir | Type | Runner |
|
||||
| ---------------------- | --------------------------------------- | --------------------------------------- |
|
||||
| `tests/unit/` | Unit tests (~500 files, fastest) | Node native test runner |
|
||||
| `tests/integration/` | Multi-module + DB integration tests | Node native test runner (concurrency 1) |
|
||||
| `tests/e2e/` | UI + workflow E2E | Playwright |
|
||||
| `tests/protocols-e2e/` | MCP + A2A real-client E2E | Custom protocol clients |
|
||||
| `tests/ecosystem/` | Provider integration (network-touching) | Node native test runner |
|
||||
| Subdir | Type | Runner |
|
||||
| ------------------------------------ | --------------------------------------- | --------------------------------------- |
|
||||
| `tests/unit/` | Unit tests (~4,800 files, fastest) | Node native test runner |
|
||||
| `tests/integration/` | Multi-module + DB integration tests | Node native test runner (concurrency 1) |
|
||||
| `tests/e2e/` | UI + workflow E2E | Playwright |
|
||||
| `tests/e2e/protocol-clients.test.ts` | MCP + A2A real-client E2E | Custom protocol clients |
|
||||
| `tests/e2e/ecosystem.test.ts` | Provider integration (network-touching) | Node native test runner |
|
||||
|
||||
---
|
||||
|
||||
@@ -505,7 +503,7 @@ Shipped configuration templates plus the committed quality-gate baselines
|
||||
| ---------------------------------- | -------------------------------------------------------------- |
|
||||
| `.github/workflows/` | GitHub Actions CI/CD workflows (lint, test, coverage, release) |
|
||||
| `.github/ISSUE_TEMPLATE/` | Bug/feature issue templates |
|
||||
| `.github/PULL_REQUEST_TEMPLATE.md` | PR template |
|
||||
| `.github/pull_request_template.md` | PR template |
|
||||
| `.github/dependabot.yml` | Dependency update config |
|
||||
|
||||
---
|
||||
|
||||
@@ -11,33 +11,33 @@ Objective feature comparison vs popular open-source AI routers.
|
||||
> **Methodology**: Public repos audited 2026-Q2. Versions as listed.
|
||||
> Submit corrections via PR — we want this to be accurate.
|
||||
|
||||
| Feature | OmniRoute 3.8 | LiteLLM 1.x | OpenRouter (SaaS) | Portkey |
|
||||
| -------------------------------------------------- | :----------------------------------------------: | :------------: | :---------------: | :---------: |
|
||||
| **Providers** | **329** | ~100 | ~50 | ~30 |
|
||||
| **Free/no-auth catalog entries** | **155** | n/a | passthrough | n/a |
|
||||
| **Self-hostable** | ✅ | ✅ | ❌ | ⚠ paid |
|
||||
| **OAuth catalog entries** | **23** | partial | ❌ | ❌ |
|
||||
| **Auto-fallback combos** | **19 strategies** | priority-based | tier-based | weighted |
|
||||
| **Fusion (parallel panel + judge synthesis)** | ✅ | ❌ | ❌ | ❌ |
|
||||
| **Tier 1/2/3 fallback (subscription→cheap→free)** | ✅ + UI | manual | n/a | manual |
|
||||
| **Token compression** | 12-engine stack (RTK + Caveman + LLMLingua) | none | none | none |
|
||||
| **Multimodal generation (speech/music/video)** | ✅ | ❌ | passthrough | ❌ |
|
||||
| **Built-in MCP server** | ✅ 107 tools, 32 scopes | ❌ | ❌ | ❌ |
|
||||
| **A2A protocol** | ✅ 6 skills | ❌ | ❌ | ❌ |
|
||||
| **Memory (FTS5 + vector)** | ✅ | ❌ | ❌ | ❌ |
|
||||
| **Guardrails (PII, injection, vision)** | ✅ | partial | ❌ | ✅ paid |
|
||||
| **Cloud agent integrations** | Codex, Cursor, Devin, Jules | ❌ | ❌ | ❌ |
|
||||
| **Circuit breaker per provider** | ✅ 3-state, lazy recovery | basic | ❌ | ✅ |
|
||||
| **TLS fingerprint stealth (JA3/JA4)** | ✅ wreq-js | ❌ | ❌ | ❌ |
|
||||
| **Eval framework** | ✅ built-in | ❌ | ❌ | ⚠ paid |
|
||||
| **MITM proxy (intercepts Cursor/Antigravity)** | ✅ cross-platform | ❌ | ❌ | ❌ |
|
||||
| **CLI with system tray (no Electron)** | ✅ | ❌ | n/a | n/a |
|
||||
| **CLI machine-ID auto-auth** | ✅ | ❌ | n/a | n/a |
|
||||
| **Dashboard** | Next.js 16 | basic | proprietary | proprietary |
|
||||
| **i18n** | **43 locales** | ❌ | ❌ | ⚠ |
|
||||
| **Public agent skills (SKILL.md)** | ✅ 45 | ❌ | ❌ | ❌ |
|
||||
| **Tunnel support (Cloudflared, Tailscale, Ngrok)** | ✅ | ❌ | n/a | n/a |
|
||||
| **License** | MIT | MIT | proprietary | proprietary |
|
||||
| Feature | OmniRoute 3.8 | LiteLLM 1.x | OpenRouter (SaaS) | Portkey |
|
||||
| -------------------------------------------------- | :-----------------------------------------: | :------------: | :---------------: | :---------: |
|
||||
| **Providers** | **329** | ~100 | ~50 | ~30 |
|
||||
| **Free/no-auth catalog entries** | **155** | n/a | passthrough | n/a |
|
||||
| **Self-hostable** | ✅ | ✅ | ❌ | ⚠ paid |
|
||||
| **OAuth catalog entries** | **23** | partial | ❌ | ❌ |
|
||||
| **Auto-fallback combos** | **19 strategies** | priority-based | tier-based | weighted |
|
||||
| **Fusion (parallel panel + judge synthesis)** | ✅ | ❌ | ❌ | ❌ |
|
||||
| **Tier 1/2/3 fallback (subscription→cheap→free)** | ✅ + UI | manual | n/a | manual |
|
||||
| **Token compression** | 12-engine stack (RTK + Caveman + LLMLingua) | none | none | none |
|
||||
| **Multimodal generation (speech/music/video)** | ✅ | ❌ | passthrough | ❌ |
|
||||
| **Built-in MCP server** | ✅ 110 tools, 33 scopes | ❌ | ❌ | ❌ |
|
||||
| **A2A protocol** | ✅ 6 skills | ❌ | ❌ | ❌ |
|
||||
| **Memory (FTS5 + vector)** | ✅ | ❌ | ❌ | ❌ |
|
||||
| **Guardrails (PII, injection, vision)** | ✅ | partial | ❌ | ✅ paid |
|
||||
| **Cloud agent integrations** | Codex, Cursor, Devin, Jules | ❌ | ❌ | ❌ |
|
||||
| **Circuit breaker per provider** | ✅ 3-state, lazy recovery | basic | ❌ | ✅ |
|
||||
| **TLS fingerprint stealth (JA3/JA4)** | ✅ wreq-js | ❌ | ❌ | ❌ |
|
||||
| **Eval framework** | ✅ built-in | ❌ | ❌ | ⚠ paid |
|
||||
| **MITM proxy (intercepts Cursor/Antigravity)** | ✅ cross-platform | ❌ | ❌ | ❌ |
|
||||
| **CLI with system tray (no Electron)** | ✅ | ❌ | n/a | n/a |
|
||||
| **CLI machine-ID auto-auth** | ✅ | ❌ | n/a | n/a |
|
||||
| **Dashboard** | Next.js 16 | basic | proprietary | proprietary |
|
||||
| **i18n** | **43 locales** | ❌ | ❌ | ⚠ |
|
||||
| **Public agent skills (SKILL.md)** | ✅ 45 | ❌ | ❌ | ❌ |
|
||||
| **Tunnel support (Cloudflared, Tailscale, Ngrok)** | ✅ | ❌ | n/a | n/a |
|
||||
| **License** | MIT | MIT | proprietary | proprietary |
|
||||
|
||||
## When to choose OmniRoute
|
||||
|
||||
|
||||
@@ -30,10 +30,10 @@ inside GitHub's `<img>` sandbox:
|
||||
| File | Used in | Notes |
|
||||
| ------------------------------------------------------ | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| [tier-cascade.svg](./tier-cascade.svg) | README.md (root) | Animated 4-tier auto-fallback cascade (16s loop, 4 acts). Edit the SVG directly — there is no `.mmd` source. |
|
||||
| [pool-fair-share.svg](./pool-fair-share.svg) | README.md (root) | Animated key-pool fair-share quota (generous → strict, 16s loop). Edit the SVG directly — there is no `.mmd` source. |
|
||||
| [pool-fair-share.svg](./pool-fair-share.svg) | — (unused) | Animated key-pool fair-share quota (generous → strict, 16s loop). Edit the SVG directly — there is no `.mmd` source. |
|
||||
| [combo-always-on.svg](./combo-always-on.svg) | style reference | Animated priority-combo fallback (4 layers, 16s loop). Edit the SVG directly — there is no `.mmd` source. |
|
||||
| [cli-terminal.svg](./cli-terminal.svg) | README.md (root) | Compact half-height animated terminal (1200×350): 3 real CLI commands cycling with typewriter + scrolling subcommand ticker; first frame = completed providers screen. Edit the SVG directly — there is no `.mmd` source. |
|
||||
| [compression-pipeline.svg](./compression-pipeline.svg) | README.md (root) | Animated 10-engine compression funnel (8s loop). Edit the SVG directly — there is no `.mmd` source. |
|
||||
| [compression-pipeline.svg](./compression-pipeline.svg) | README.md (root) | Animated 12-engine compression funnel (8s loop). Edit the SVG directly — there is no `.mmd` source. |
|
||||
| [free-tier-budget.svg](./free-tier-budget.svg) | README.md (root) | Animated free-tier budget card (~1.51B/mo quantified headline, 20-pool budget bar, per-pool grid, signup credits, 10s loop). Edit the SVG directly — there is no `.mmd` source. |
|
||||
| [readme-hero.svg](./readme-hero.svg) | README.md (root) | Animated hero card (tagline, live provider/free-access headline, full-width compression bar demo, 6 stat chips). Edit the SVG directly — there is no `.mmd` source. |
|
||||
| [promise-pillars.svg](./promise-pillars.svg) | README.md (root) | Animated "The Promise" 6-pillar card (12s border-highlight sweep). Edit the SVG directly — there is no `.mmd` source. |
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
<svg viewBox="0 0 1200 350" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="Animated terminal demoing the OmniRoute CLI: omniroute providers list (352 providers registered, anthropic, codex, glm, kimi shown active), omniroute combo list (always-on priority, cost-saver, fusion-panel, context-relay) and omniroute health (healthy, 18412 requests in 24h, p95 412ms, circuit breakers 24 closed, 1 half-open, 0 open), cycling over 85 top-level commands: providers, oauth, keys, combo, nodes, models, cache, compression, cost, usage, quota, health, resilience, telemetry, logs, audit, mcp, a2a, cloud, memory, skills, eval, doctor, repl, tunnel, backup, sync, webhooks, policy, pricing, translator, simulate and more.">
|
||||
<svg viewBox="0 0 1200 350" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="Animated terminal demoing the OmniRoute CLI: omniroute providers list (352 providers registered, anthropic, codex, glm, kimi shown active), omniroute combo list (always-on priority, cost-saver, fusion-panel, context-relay) and omniroute health (healthy, 18412 requests in 24h, p95 412ms, circuit breakers 24 closed, 1 half-open, 0 open), cycling over 86 top-level commands: providers, oauth, keys, combo, nodes, models, cache, compression, cost, usage, quota, health, resilience, telemetry, logs, audit, mcp, a2a, cloud, memory, skills, eval, doctor, repl, tunnel, backup, sync, webhooks, policy, pricing, translator, simulate and more.">
|
||||
<desc>Compact animated terminal cycling three real OmniRoute CLI commands with a typewriter effect and a scrolling subcommand ticker; the first frame shows the completed providers-list screen.</desc>
|
||||
<defs><clipPath id="tickerClip"><rect x="12" y="304" width="1176" height="40"/></clipPath><clipPath id="tw0"><rect x="64" y="46" height="26" width="0"><animate attributeName="width" calcMode="discrete" values="0;31;61;92;122;153;184;214;245;245" keyTimes="0;0.012;0.018;0.024;0.030;0.036;0.042;0.048;0.054;1" dur="18s" repeatCount="indefinite"/></rect></clipPath><clipPath id="tw1"><rect x="64" y="46" height="26" width="0"><animate attributeName="width" calcMode="discrete" values="0;26;51;76;102;128;153;178;204;204" keyTimes="0;0.348;0.351;0.357;0.363;0.369;0.375;0.381;0.387;1" dur="18s" repeatCount="indefinite"/></rect></clipPath><clipPath id="tw2"><rect x="64" y="46" height="26" width="0"><animate attributeName="width" calcMode="discrete" values="0;20;41;61;82;102;122;143;163;163" keyTimes="0;0.678;0.684;0.690;0.696;0.702;0.708;0.714;0.720;1" dur="18s" repeatCount="indefinite"/></rect></clipPath></defs>
|
||||
<rect width="1200" height="350" fill="#0d1117"/>
|
||||
<rect x="0" y="0" width="1200" height="34" fill="#161b22"/>
|
||||
<path d="M 0 34 L 1200 34" stroke="#ffffff" stroke-opacity="0.08" stroke-width="1"/>
|
||||
<circle cx="24" cy="17" r="6" fill="#ff5f56"/><circle cx="46" cy="17" r="6" fill="#ffbd2e"/><circle cx="68" cy="17" r="6" fill="#27c93f"/>
|
||||
<text x="600" y="22" text-anchor="middle" font-family="Consolas, 'Courier New', monospace" font-size="13" fill="#71717a">omniroute — 85 top-level commands</text>
|
||||
<g font-family="Consolas, 'Courier New', monospace" font-size="17"><animate attributeName="opacity" values="1;0;0" keyTimes="0;0.006;1" dur="18s" repeatCount="indefinite"/><text x="64" y="66" fill="#F7F6FC">omniroute providers list</text><text x="40" y="100" font-weight="700" fill="#38bdf8">OmniRoute Providers</text><text x="40" y="128" fill="#a1a1aa">1f3a9c2e  anthropic   Claude Max 20x    <tspan fill='#22c55e'>active</tspan></text><text x="40" y="154" fill="#a1a1aa">8c2d5b1a  codex       Codex Pro (team)  <tspan fill='#22c55e'>active</tspan></text><text x="40" y="180" fill="#a1a1aa">f4e0a97b  glm         GLM Coding Plan   <tspan fill='#22c55e'>active</tspan></text><text x="40" y="206" fill="#a1a1aa">03bd6e5f  kimi        Kimi K2 free      <tspan fill='#22c55e'>active</tspan></text><text x="40" y="232" fill="#71717a">… 347 more providers</text></g><g opacity="1" font-family="Consolas, 'Courier New', monospace" font-size="17">
|
||||
<text x="600" y="22" text-anchor="middle" font-family="Consolas, 'Courier New', monospace" font-size="13" fill="#71717a">omniroute — 86 top-level commands</text>
|
||||
<g font-family="Consolas, 'Courier New', monospace" font-size="17"><animate attributeName="opacity" values="1;0;0" keyTimes="0;0.006;1" dur="18s" repeatCount="indefinite"/><text x="64" y="66" fill="#F7F6FC">omniroute providers list</text><text x="40" y="100" font-weight="700" fill="#38bdf8">OmniRoute Providers</text><text x="40" y="128" fill="#a1a1aa">1f3a9c2e  anthropic   Claude Max 20x    <tspan fill='#22c55e'>active</tspan></text><text x="40" y="154" fill="#a1a1aa">8c2d5b1a  codex       Codex Pro (team)  <tspan fill='#22c55e'>active</tspan></text><text x="40" y="180" fill="#a1a1aa">f4e0a97b  glm         GLM Coding Plan   <tspan fill='#22c55e'>active</tspan></text><text x="40" y="206" fill="#a1a1aa">03bd6e5f  kimi        Kimi K2 free      <tspan fill='#22c55e'>active</tspan></text><text x="40" y="232" fill="#71717a">… 348 more providers</text></g><g opacity="1" font-family="Consolas, 'Courier New', monospace" font-size="17">
|
||||
<animate attributeName="opacity" values="1;1;0;0" keyTimes="0;0.315;0.33;1" dur="18s" repeatCount="indefinite"/>
|
||||
<text x="40" y="66" fill="#22c55e">$</text>
|
||||
<g clip-path="url(#tw0)"><text x="64" y="66" fill="#F7F6FC">omniroute providers list</text></g>
|
||||
@@ -14,7 +14,7 @@
|
||||
<animate attributeName="x" calcMode="discrete" values="64;95;125;156;186;217;248;278;309;309" keyTimes="0.000;0.012;0.018;0.024;0.030;0.036;0.042;0.048;0.054;1" dur="18s" repeatCount="indefinite"/>
|
||||
<animate attributeName="opacity" values="0;0;1;0.2;1;0.2;1;0;0" keyTimes="0;0.011;0.012;0.022;0.032;0.042;0.052;0.074;1" dur="18s" repeatCount="indefinite"/>
|
||||
</rect>
|
||||
<text x="40" y="100" font-weight="700" fill="#38bdf8" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.045;0.047" dur="18s" repeatCount="indefinite"/>OmniRoute Providers</text><text x="40" y="128" fill="#a1a1aa" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.053;0.055" dur="18s" repeatCount="indefinite"/>1f3a9c2e  anthropic   Claude Max 20x    <tspan fill='#22c55e'>active</tspan></text><text x="40" y="154" fill="#a1a1aa" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.061;0.063" dur="18s" repeatCount="indefinite"/>8c2d5b1a  codex       Codex Pro (team)  <tspan fill='#22c55e'>active</tspan></text><text x="40" y="180" fill="#a1a1aa" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.069;0.07100000000000001" dur="18s" repeatCount="indefinite"/>f4e0a97b  glm         GLM Coding Plan   <tspan fill='#22c55e'>active</tspan></text><text x="40" y="206" fill="#a1a1aa" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.077;0.079" dur="18s" repeatCount="indefinite"/>03bd6e5f  kimi        Kimi K2 free      <tspan fill='#22c55e'>active</tspan></text><text x="40" y="232" fill="#71717a" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.085;0.08700000000000001" dur="18s" repeatCount="indefinite"/>… 347 more providers</text>
|
||||
<text x="40" y="100" font-weight="700" fill="#38bdf8" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.045;0.047" dur="18s" repeatCount="indefinite"/>OmniRoute Providers</text><text x="40" y="128" fill="#a1a1aa" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.053;0.055" dur="18s" repeatCount="indefinite"/>1f3a9c2e  anthropic   Claude Max 20x    <tspan fill='#22c55e'>active</tspan></text><text x="40" y="154" fill="#a1a1aa" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.061;0.063" dur="18s" repeatCount="indefinite"/>8c2d5b1a  codex       Codex Pro (team)  <tspan fill='#22c55e'>active</tspan></text><text x="40" y="180" fill="#a1a1aa" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.069;0.07100000000000001" dur="18s" repeatCount="indefinite"/>f4e0a97b  glm         GLM Coding Plan   <tspan fill='#22c55e'>active</tspan></text><text x="40" y="206" fill="#a1a1aa" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.077;0.079" dur="18s" repeatCount="indefinite"/>03bd6e5f  kimi        Kimi K2 free      <tspan fill='#22c55e'>active</tspan></text><text x="40" y="232" fill="#71717a" opacity="0"><animate attributeName="opacity" calcMode="discrete" values="0;0;1" keyTimes="0;0.085;0.08700000000000001" dur="18s" repeatCount="indefinite"/>… 348 more providers</text>
|
||||
</g><g opacity="0" font-family="Consolas, 'Courier New', monospace" font-size="17">
|
||||
<animate attributeName="opacity" values="0;0;1;1;0;0" keyTimes="0;0.333;0.34800000000000003;0.648;0.663;1" dur="18s" repeatCount="indefinite"/>
|
||||
<text x="40" y="66" fill="#22c55e">$</text>
|
||||
|
||||
|
Before Width: | Height: | Size: 12 KiB After Width: | Height: | Size: 12 KiB |
@@ -1,4 +1,4 @@
|
||||
<svg viewBox="0 0 1200 780" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="Comparison table: OmniRoute versus 9router, OpenRouter, CLIProxyAPI and LiteLLM across 13 capabilities. OmniRoute is the only one with the full set: 352 providers, 90+ free providers built-in, 19 routing strategies, 12-engine token compression, a built-in MCP server with 110 tools, A2A protocol, persistent memory, guardrails, cloud agents, TLS fingerprint stealth, desktop/Termux/PWA, 43 UI locales and 100% MIT self-hosted. 9router has free providers, RTK compression and translation but no MCP, A2A, memory, guardrails, cloud agents or stealth. OpenRouter is a hosted SaaS with 400+ models, guardrails and a hosted MCP but is not self-hosted and lacks A2A, memory, cloud agents and stealth. CLIProxyAPI is a light OAuth proxy with two routing strategies. LiteLLM has 100+ providers, A2A and extensive guardrails but no memory, compression, free tier, stealth or cloud agents.">
|
||||
<svg viewBox="0 0 1200 780" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="Comparison table: OmniRoute versus 9router, OpenRouter, CLIProxyAPI and LiteLLM across 13 capabilities. OmniRoute is the only one with the full set: 352 providers, 150+ free providers built-in, 19 routing strategies, 12-engine token compression, a built-in MCP server with 110 tools, A2A protocol, persistent memory, guardrails, cloud agents, TLS fingerprint stealth, desktop/Termux/PWA, 43 UI locales and 100% MIT self-hosted. 9router has free providers, RTK compression and translation but no MCP, A2A, memory, guardrails, cloud agents or stealth. OpenRouter is a hosted SaaS with 400+ models, guardrails and a hosted MCP but is not self-hosted and lacks A2A, memory, cloud agents and stealth. CLIProxyAPI is a light OAuth proxy with two routing strategies. LiteLLM has 100+ providers, A2A and extensive guardrails but no memory, compression, free tier, stealth or cloud agents.">
|
||||
<desc>Static-header comparison table where each capability row fades in top to bottom; the OmniRoute column is highlighted and shows a check or a leading value in every row, while competitors show a mix of checks, partials and crosses.</desc>
|
||||
<defs>
|
||||
<pattern id="gC" width="32" height="32" patternUnits="userSpaceOnUse"><path d="M 32 0 L 0 0 0 32" fill="none" stroke="#ffffff" stroke-opacity="0.05" stroke-width="1"/></pattern>
|
||||
@@ -23,7 +23,7 @@
|
||||
<g font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif">
|
||||
<g opacity="0"><animate attributeName="opacity" values="0;1" dur="0.4s" begin="0.15s" fill="freeze"/>
|
||||
<text x="44" y="196" font-size="14.5" fill="#c9d1d9">Providers</text>
|
||||
<text x="440" y="196" text-anchor="middle" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="15" font-weight="800" fill="#7ee787">350</text>
|
||||
<text x="440" y="196" text-anchor="middle" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="15" font-weight="800" fill="#7ee787">352</text>
|
||||
<text x="604" y="196" text-anchor="middle" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="13.5" font-weight="600" fill="#8b949e">40+</text>
|
||||
<text x="760" y="196" text-anchor="middle" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="13.5" font-weight="600" fill="#8b949e">400+*</text>
|
||||
<text x="916" y="196" text-anchor="middle" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="13.5" font-weight="600" fill="#8b949e">~5</text>
|
||||
|
||||
|
Before Width: | Height: | Size: 13 KiB After Width: | Height: | Size: 13 KiB |
@@ -1,5 +1,5 @@
|
||||
%% Database schema overview (selected core tables)
|
||||
%% Reflects: src/lib/db/* (95+ modules, 110+ migrations)
|
||||
%% Reflects: src/lib/db/* (120+ modules, 167 migrations)
|
||||
%% v3.8.0
|
||||
erDiagram
|
||||
api_keys ||--o{ api_key_usage : tracks
|
||||
|
||||
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 24 KiB After Width: | Height: | Size: 24 KiB |
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 25 KiB After Width: | Height: | Size: 25 KiB |
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 40 KiB After Width: | Height: | Size: 40 KiB |
@@ -1,4 +1,4 @@
|
||||
<svg viewBox="0 0 1200 842" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="OmniRoute free-tier budget: about 1.51 billion free tokens per month steady, up to about 2.13 billion in the first month with signup credits. The catalog contains 445 rows, 438 active and 7 discontinued, grouped into 39 recurring pool keys; 20 pools have a published positive monthly token budget and 19 have a zero, uncapped, or keyless budget. Honest pool-deduped math counts each shared free pool once; 13 providers carry a terms-of-service avoid flag. The 20 quantified pools are Mistral 1 billion, LLM7 150 million, Nara 150 million, Gemini 60 million, Cerebras 30 million, Cloudflare AI 30 million, API Airforce 24 million, Ollama Cloud 20 million, Groq 15 million, Bluesminds 7.2 million, SambaNova 6 million, Arcee 4.8 million, Navy 4.5 million, BazaarLink 3.6 million, OpenRouter 1.2 million, Cohere 800 thousand, HuggingChat 500 thousand, Morph 400 thousand, Hugging Face 200 thousand, and Kiro 25 thousand. One-time signup credits add about 626 million. Uncapped providers and the OpenRouter top-up boost are shown separately so they do not inflate the headline. Live usage remains available at /dashboard/free-tiers.">
|
||||
<svg viewBox="0 0 1200 842" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="OmniRoute free-tier budget: about 1.51 billion free tokens per month steady, up to about 2.13 billion in the first month with signup credits. The catalog contains 446 rows, 439 active and 7 discontinued, grouped into 38 recurring pool keys; 20 pools have a published positive monthly token budget and 18 have a zero, uncapped, or keyless budget. Honest pool-deduped math counts each shared free pool once; 13 providers carry a terms-of-service avoid flag. The 20 quantified pools are Mistral 1 billion, LLM7 150 million, Nara 150 million, Gemini 60 million, Cerebras 30 million, Cloudflare AI 30 million, API Airforce 24 million, Ollama Cloud 20 million, Groq 15 million, Bluesminds 7.2 million, SambaNova 6 million, Arcee 4.8 million, Navy 4.5 million, BazaarLink 3.6 million, OpenRouter 1.2 million, Cohere 800 thousand, HuggingChat 500 thousand, Morph 400 thousand, Hugging Face 200 thousand, and Kiro 25 thousand. One-time signup credits add about 626 million. Uncapped providers and the OpenRouter top-up boost are shown separately so they do not inflate the headline. Live usage remains available at /dashboard/free-tiers.">
|
||||
<desc>Pool-deduplicated chart of the 20 recurring free-token pools with positive published budgets, plus signup credits and uncapped providers shown separately.</desc>
|
||||
<defs>
|
||||
<pattern id="gridPaperF" width="32" height="32" patternUnits="userSpaceOnUse">
|
||||
@@ -64,7 +64,7 @@
|
||||
<text x="60" y="228" font-family="Consolas, 'Courier New', monospace" font-size="104" font-weight="800" fill="url(#gradBrandF)">~1.51B</text>
|
||||
<text x="62" y="266" font-family="Consolas, 'Courier New', monospace" font-size="15" letter-spacing="3" font-weight="700" fill="#a1a1aa">FREE TOKENS / MONTH · <tspan fill="#22c55e">STEADY</tspan></text>
|
||||
<text x="62" y="298" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="16" fill="#F7F6FC">up to <tspan font-weight="800" fill="#22c55e">~2.13B</tspan> in your first month — signup credits</text>
|
||||
<text x="62" y="326" font-family="Consolas, 'Courier New', monospace" font-size="12" fill="#71717a">documented free tiers · <tspan fill="#8b5cf6">39 recurring pools</tspan> · <tspan fill="#8b5cf6">445 catalog entries</tspan> · one endpoint</text>
|
||||
<text x="62" y="326" font-family="Consolas, 'Courier New', monospace" font-size="12" fill="#71717a">documented free tiers · <tspan fill="#8b5cf6">38 recurring pools</tspan> · <tspan fill="#8b5cf6">446 catalog entries</tspan> · one endpoint</text>
|
||||
|
||||
<!-- ═══ Panel · The honest math ═══ -->
|
||||
<rect x="680" y="84" width="460" height="216" rx="14" fill="#161b22" stroke="#ffffff" stroke-opacity="0.08" stroke-width="1"/>
|
||||
@@ -78,7 +78,7 @@
|
||||
<text x="704" y="240" font-family="Consolas, 'Courier New', monospace" font-size="34" font-weight="800" fill="#22c55e">~1.51B</text>
|
||||
<text x="836" y="224" font-family="Consolas, 'Courier New', monospace" font-size="11.5" fill="#a1a1aa">each shared free pool</text>
|
||||
<text x="836" y="244" font-family="Consolas, 'Courier New', monospace" font-size="11.5" fill="#22c55e">counted once ✓</text>
|
||||
<text x="704" y="280" font-family="Consolas, 'Courier New', monospace" font-size="12" fill="#f59e0b"><tspan font-weight="800">14 providers</tspan> ToS-flagged <tspan fill="#71717a">— we flag it · you decide</tspan></text>
|
||||
<text x="704" y="280" font-family="Consolas, 'Courier New', monospace" font-size="12" fill="#f59e0b"><tspan font-weight="800">13 providers</tspan> ToS-flagged <tspan fill="#71717a">— we flag it · you decide</tspan></text>
|
||||
|
||||
<!-- ═══ Budget bar · 20 quantified recurring pools ═══ -->
|
||||
<text x="60" y="356" font-family="Consolas, 'Courier New', monospace" font-size="10.5" letter-spacing="2.5" font-weight="700" fill="#a78bfa">WHERE IT COMES FROM · <tspan fill="#F7F6FC">20 QUANTIFIED RECURRING POOLS</tspan></text>
|
||||
|
||||
|
Before Width: | Height: | Size: 18 KiB After Width: | Height: | Size: 18 KiB |
@@ -5,7 +5,7 @@ flowchart LR
|
||||
Src["Source MDs<br/>(CLAUDE.md, docs/**/*.md)"] --> Hash["sha256 hash"]
|
||||
Hash --> State[".i18n-state.json"]
|
||||
State -->|diff| Dirty["Mark source dirty"]
|
||||
Dirty --> Loop{"For each locale<br/>(39 langs)"}
|
||||
Dirty --> Loop{"For each locale<br/>(43 langs)"}
|
||||
Loop --> LLM["OmniRoute<br/>/chat/completions<br/>(cx/gpt-5.4-mini)"]
|
||||
LLM --> Target["Write<br/>docs/i18n/<locale>/<rel-path>.md"]
|
||||
Target --> State
|
||||
|
||||
@@ -1,14 +1,14 @@
|
||||
%% MCP Server tool inventory by category
|
||||
%% Reflects: open-sse/mcp-server/ — TOTAL_MCP_TOOL_COUNT in server.ts (countUniqueMcpTools)
|
||||
%% Unique contributions: MCP_TOOLS 42 + memory 3 + skill 4 + githubSkill 3 + pool 6
|
||||
%% + gamification 8 + plugin 8 + notion 6 + obsidian 22 + localCorpus 3 + compression 2 = 107
|
||||
%% Unique contributions: MCP_TOOLS 45 + memory 3 + skill 4 + githubSkill 3 + pool 6
|
||||
%% + gamification 8 + plugin 8 + notion 6 + obsidian 22 + localCorpus 3 + compression 2 = 110
|
||||
%% (agentSkillTools fully overlaps the base registry; 11 of the 13 compression
|
||||
%% tools live in the base registry — duplicates counted once)
|
||||
%% Regenerate the SVG with: npm run docs:render-diagrams
|
||||
%% v3.8.50
|
||||
%% v3.8.51
|
||||
flowchart LR
|
||||
MCP["MCP Server<br/>107 tools total"]
|
||||
MCP --> Core["Base registry (42)<br/>routing, cache, compression, quota,<br/>proxy, web search/fetch, tool search"]
|
||||
MCP["MCP Server<br/>110 tools total"]
|
||||
MCP --> Core["Base registry (45)<br/>routing, cache, compression, quota,<br/>proxy, web search/fetch, tool search"]
|
||||
MCP --> Mem["Memory (3)<br/>memory_search, memory_add,<br/>memory_clear"]
|
||||
MCP --> Skl["Skills (4)<br/>skills_list, skills_enable,<br/>skills_execute, skills_executions<br/>+ agent-skill catalog discovery"]
|
||||
MCP --> GSkl["GitHub-Skills (3)<br/>search, scan, install"]
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
<svg viewBox="0 0 1200 540" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="The OmniRoute promise: one endpoint and 352 providers. Six pillars. Resilient fallback: automatic routing continues while another healthy target is available. Save up to 95 percent of eligible tokens: RTK plus Caveman stacked compression averages about 89 percent on tool-heavy sessions. Zero dollars to start: 90+ providers with a free tier and 56 recurring or keyless free-forever providers. Every tool works: 35 CLI and agent integration records, including Claude Code, Codex, Cursor, Cline, Copilot and Antigravity, through one config. One endpoint: OpenAI, Claude, Gemini and Responses API translation at /v1. Production controls: circuit breakers, TLS stealth, MCP with 110 tools, A2A, memory, guardrails, evals, and 39,000+ static test declarations across 5,100+ tracked test files.">
|
||||
<svg viewBox="0 0 1200 540" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="The OmniRoute promise: one endpoint and 352 providers. Six pillars. Resilient fallback: automatic routing continues while another healthy target is available. Save up to 95 percent of eligible tokens: RTK plus Caveman stacked compression averages about 89 percent on tool-heavy sessions. Zero dollars to start: 150+ providers with a free tier and 53 recurring or keyless free-forever providers. Every tool works: 36 CLI and agent integration records, including Claude Code, Codex, Cursor, Cline, Copilot and Antigravity, through one config. One endpoint: OpenAI, Claude, Gemini and Responses API translation at /v1. Production controls: circuit breakers, TLS stealth, MCP with 110 tools, A2A, memory, guardrails, evals, and 39,000+ static test declarations across 5,100+ tracked test files.">
|
||||
<desc>Animated promise card: six pillar tiles fade in in reading order, then a soft colored border highlight sweeps from tile to tile in a continuous cycle.</desc>
|
||||
<defs>
|
||||
<pattern id="gridPaperP" width="32" height="32" patternUnits="userSpaceOnUse">
|
||||
@@ -73,7 +73,7 @@
|
||||
<circle cx="6.6" cy="6.6" r="1.4" fill="#fdcb6e" stroke="none"/>
|
||||
</g>
|
||||
<text x="862" y="170" font-size="18" font-weight="800" fill="#fdcb6e">$0 to start</text>
|
||||
<text x="826" y="204" font-size="13.5" fill="#a1a1aa">90+ providers with a free tier, 55 free</text>
|
||||
<text x="826" y="204" font-size="13.5" fill="#a1a1aa">150+ providers with a free tier, 53 free</text>
|
||||
<text x="826" y="226" font-size="13.5" fill="#a1a1aa">forever — Qoder, Pollinations, Cloudflare,</text>
|
||||
<text x="826" y="248" font-size="13.5" fill="#a1a1aa">SiliconFlow… No card needed.</text>
|
||||
</g>
|
||||
@@ -91,7 +91,7 @@
|
||||
<path d="M 10,18 L 10,22"/>
|
||||
</g>
|
||||
<text x="102" y="354" font-size="18" font-weight="800" fill="#a78bfa">Every tool works</text>
|
||||
<text x="66" y="388" font-size="13.5" fill="#a1a1aa">35 CLI/agent integrations — Claude Code, Codex,</text>
|
||||
<text x="66" y="388" font-size="13.5" fill="#a1a1aa">36 CLI/agent integrations — Claude Code, Codex,</text>
|
||||
<text x="66" y="410" font-size="13.5" fill="#a1a1aa">Cursor, Cline, Copilot, Antigravity —</text>
|
||||
<text x="66" y="432" font-size="13.5" fill="#a1a1aa">through one config.</text>
|
||||
</g>
|
||||
|
||||
|
Before Width: | Height: | Size: 10 KiB After Width: | Height: | Size: 10 KiB |
@@ -1,4 +1,4 @@
|
||||
<svg viewBox="0 0 1200 548" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="OmniRoute hero: Never stop coding. Every AI tool to 352 providers — 90+ free — through one endpoint. Claude Code, Codex, Cursor, Cline, Copilot and Antigravity into free Claude, GPT and Gemini with auto-fallback. RTK + Caveman stacked compression saves 15 to 95 percent of tokens — about 89 percent average on tool-heavy sessions — so you never hit limits. Stats: 352 AI providers, 90+ free tiers, about 1.51B free tokens per month, 15 to 95 percent token savings, 19 routing strategies, zero dollars to start.">
|
||||
<svg viewBox="0 0 1200 548" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="OmniRoute hero: Never stop coding. Every AI tool to 352 providers — 150+ free — through one endpoint. Claude Code, Codex, Cursor, Cline, Copilot and Antigravity into free Claude, GPT and Gemini with auto-fallback. RTK + Caveman stacked compression saves 15 to 95 percent of tokens — about 89 percent average on tool-heavy sessions — so you never hit limits. Stats: 352 AI providers, 150+ free tiers, about 1.51B free tokens per month, 15 to 95 percent token savings, 19 routing strategies, zero dollars to start.">
|
||||
<desc>Animated hero card: a pulse travels the divider line and a compression bar demo repeatedly shrinks a prompt by up to 95 percent; all headline content is static and readable on the first frame.</desc>
|
||||
<defs>
|
||||
<pattern id="gridPaperH" width="32" height="32" patternUnits="userSpaceOnUse">
|
||||
@@ -28,7 +28,7 @@
|
||||
<text x="48" y="138" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="60" font-weight="800" fill="#e9edf3">Never stop coding<tspan fill="#a855f7">.</tspan></text>
|
||||
|
||||
<!-- subheadline -->
|
||||
<text x="48" y="184" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="25" font-weight="600" fill="#c9d1d9">Every AI tool → <tspan fill="#a78bfa" font-weight="800">352 providers</tspan> — <tspan fill="#7ee787" font-weight="800">90+ free</tspan> — through one endpoint.</text>
|
||||
<text x="48" y="184" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="25" font-weight="600" fill="#c9d1d9">Every AI tool → <tspan fill="#a78bfa" font-weight="800">352 providers</tspan> — <tspan fill="#7ee787" font-weight="800">150+ free</tspan> — through one endpoint.</text>
|
||||
|
||||
<!-- plug line -->
|
||||
<text x="48" y="222" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="16.5" fill="#a1a1aa">Claude Code · Codex · Cursor · Cline · Copilot · Antigravity  →  <tspan fill="#7ee787" font-weight="700">FREE</tspan> Claude / GPT / Gemini · auto-fallback</text>
|
||||
@@ -66,7 +66,7 @@
|
||||
<!-- stat chips -->
|
||||
<g font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" text-anchor="middle">
|
||||
<rect x="48" y="448" width="172" height="52" rx="12" fill="#161b22" stroke="#6c5ce7" stroke-opacity="0.55" stroke-width="1.5"/>
|
||||
<text x="134" y="471" font-size="17" font-weight="800" fill="#a78bfa">350</text>
|
||||
<text x="134" y="471" font-size="17" font-weight="800" fill="#a78bfa">352</text>
|
||||
<text x="134" y="490" font-size="11" fill="#a1a1aa">AI PROVIDERS</text>
|
||||
<rect x="234" y="448" width="172" height="52" rx="12" fill="#161b22" stroke="#22c55e" stroke-opacity="0.55" stroke-width="1.5"/>
|
||||
<text x="320" y="471" font-size="17" font-weight="800" fill="#7ee787">90+</text>
|
||||
|
||||
|
Before Width: | Height: | Size: 7.3 KiB After Width: | Height: | Size: 7.3 KiB |
@@ -17,8 +17,8 @@ flowchart LR
|
||||
Resolve --> Single["handleSingleModel<br/>(per target)"]
|
||||
Combo -->|single| Single
|
||||
Single --> Translate["translateRequest<br/>(OpenAI↔Claude↔Gemini)"]
|
||||
Translate --> Exec["getExecutor<br/>(89 executor modules)"]
|
||||
Exec --> Upstream["Upstream Provider<br/>(329 catalog entries)"]
|
||||
Translate --> Exec["getExecutor<br/>(104 executor modules)"]
|
||||
Exec --> Upstream["Upstream Provider<br/>(352 catalog entries)"]
|
||||
Upstream --> Stream["SSE / JSON"]
|
||||
Stream --> Transformer["responsesTransformer<br/>(Responses↔Chat)"]
|
||||
Transformer --> Client
|
||||
|
||||
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 22 KiB After Width: | Height: | Size: 22 KiB |
@@ -141,16 +141,16 @@ curl -X POST http://localhost:20128/a2a \
|
||||
|
||||
OmniRoute exposes 6 A2A skills wired in `src/lib/a2a/taskExecution.ts::A2A_SKILL_HANDLERS`. Each skill module lives in `src/lib/a2a/skills/`.
|
||||
|
||||
| Skill | ID | Description | Tags | Examples |
|
||||
| :----------------- | :------------------- | :-------------------------------------------------------------------------------------------------------------- | :------------------------- | :------------------------------------- |
|
||||
| Smart Routing | `smart-routing` | Routes a prompt through the optimal provider/combo using OmniRoute's combo engine + scoring | routing, providers | "Route this prompt via the best model" |
|
||||
| Quota Management | `quota-management` | Reports per-provider quota state, helps callers decide when to throttle/switch | quota, providers | "Check quota for anthropic" |
|
||||
| Provider Discovery | `provider-discovery` | Lists installed providers with capabilities, free-tier flags, OAuth status | providers, discovery | "What providers are available?" |
|
||||
| Cost Analysis | `cost-analysis` | Estimates cost of a request/conversation given the catalog + recent usage | cost, usage | "Estimate cost for this conversation" |
|
||||
| Health Report | `health-report` | Aggregates circuit breaker, cooldown, lockout state per provider | health, resilience | "Show health status of all providers" |
|
||||
| Skill | ID | Description | Tags | Examples |
|
||||
| :----------------- | :------------------- | :------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------- | :------------------------------------- |
|
||||
| Smart Routing | `smart-routing` | Routes a prompt through the optimal provider/combo using OmniRoute's combo engine + scoring | routing, providers | "Route this prompt via the best model" |
|
||||
| Quota Management | `quota-management` | Reports per-provider quota state, helps callers decide when to throttle/switch | quota, providers | "Check quota for anthropic" |
|
||||
| Provider Discovery | `provider-discovery` | Lists installed providers with capabilities, free-tier flags, OAuth status | providers, discovery | "What providers are available?" |
|
||||
| Cost Analysis | `cost-analysis` | Estimates cost of a request/conversation given the catalog + recent usage | cost, usage | "Estimate cost for this conversation" |
|
||||
| Health Report | `health-report` | Aggregates circuit breaker, cooldown, lockout state per provider | health, resilience | "Show health status of all providers" |
|
||||
| List Capabilities | `list-capabilities` | Returns the full 45-entry Agent Skills catalog (23 API + 21 CLI + 1 config) as a markdown table with raw SKILL.md URLs for context injection | catalog, discovery, skills | "List all OmniRoute capabilities" |
|
||||
|
||||
> The Agent Card should be kept aligned with the live 329-provider catalog; provider counts and free/no-auth metadata are sourced from the runtime registry.
|
||||
> The Agent Card should be kept aligned with the live 352-provider catalog; provider counts and free/no-auth metadata are sourced from the runtime registry.
|
||||
|
||||
### `list-capabilities` Skill Detail
|
||||
|
||||
@@ -163,7 +163,7 @@ The `list-capabilities` skill is particularly useful for external agents that ne
|
||||
...
|
||||
```
|
||||
|
||||
Each row includes the `rawUrl` column so agents can immediately fetch the full SKILL.md. The `metadata.totalSkills` field is always `42`. Implementation: `src/lib/a2a/skills/listCapabilities.ts`. See also [AGENT-SKILLS.md](./AGENT-SKILLS.md).
|
||||
Each row includes the `rawUrl` column so agents can immediately fetch the full SKILL.md. The `metadata.totalSkills` field mirrors the catalog size (45 today). Implementation: `src/lib/a2a/skills/listCapabilities.ts`. See also [AGENT-SKILLS.md](./AGENT-SKILLS.md).
|
||||
|
||||
---
|
||||
|
||||
@@ -171,13 +171,13 @@ Each row includes the `rawUrl` column so agents can immediately fetch the full S
|
||||
|
||||
The JSON-RPC endpoint `/a2a` is the canonical A2A entry point. The REST endpoints below provide auxiliary access for dashboards and external tooling:
|
||||
|
||||
| Endpoint | Method | Description | Auth |
|
||||
| :--------------------------- | :----- | :------------------------------- | :--------------------- |
|
||||
| `/api/a2a/status` | GET | Server status, registered skills | (public) |
|
||||
| `/api/a2a/tasks` | GET | List tasks with filters | management |
|
||||
| `/api/a2a/tasks/[id]` | GET | Get task by ID | management |
|
||||
| `/api/a2a/tasks/[id]/cancel` | POST | Cancel running task | management |
|
||||
| `/.well-known/agent.json` | GET | Agent Card (A2A discovery) | (public, cached 3600s) |
|
||||
| Endpoint | Method | Description | Auth |
|
||||
| :--------------------------- | :----- | :---------------------------------------------------------------- | :------------------------------------------- |
|
||||
| `/api/a2a/status` | GET | Server status, registered skills | (public) |
|
||||
| `/api/a2a/tasks` | GET | List tasks with filters | management |
|
||||
| `/api/a2a/tasks/[id]` | GET | Get task by ID | management |
|
||||
| `/api/a2a/tasks/[id]/cancel` | POST | Cancel running task | management |
|
||||
| `/.well-known/agent.json` | GET | Agent Card (A2A discovery) | (public, cached 3600s) |
|
||||
| `/api/a2a/tasks` | POST | Inbound delegation to the OmniConductor fleet (Conductor PRD RF5) | Bearer vs `OMNIROUTE_API_KEY` + `a2aEnabled` |
|
||||
|
||||
**Inbound Conductor delegation (`POST /api/a2a/tasks`):** external A2A agents delegate coding work to the OmniConductor fleet through OmniRoute. Body: `{ skill: "conductor" | "conductor-cli-<profile>", messages: [{role, content}], metadata: { conductor: { repo: { url, base_ref? }, mode?, cli?, model? } } }` — only Conductor fleet skills (the ones announced on the Agent Card) are delegable; `metadata.conductor.repo.url` is required (the fleet works on git repos). The route translates to the hub's `POST /v1/tasks` using the server-side `CONDUCTOR_ORCHESTRATOR_TOKEN` (fallback `CONDUCTOR_HUB_TOKEN`) and returns `201 { conductor_task_id, state: "submitted" }`; task states flow back through the SSE→A2A mirror (RF1) and are visible via `GET /api/a2a/tasks?skill=conductor`.
|
||||
|
||||
@@ -19,7 +19,7 @@ ACP (Agent Client Protocol) is a **"CLI-as-backend" transport** for OmniRoute. I
|
||||
| **No API keys needed** | Uses your existing CLI authentication |
|
||||
| **Native protocol** | Uses each CLI's native input/output format |
|
||||
| **Auto-discovery** | Detects installed CLIs on your system |
|
||||
| **13 built-in agents** | Pre-configured for popular CLI tools |
|
||||
| **15 built-in agents** | Pre-configured for popular CLI tools |
|
||||
| **Custom agents** | Add your own CLI tools via settings |
|
||||
| **Process management** | Handles lifecycle (spawn, send, kill) |
|
||||
|
||||
@@ -27,7 +27,7 @@ ACP (Agent Client Protocol) is a **"CLI-as-backend" transport** for OmniRoute. I
|
||||
|
||||
## Supported CLI Agents
|
||||
|
||||
ACP supports **13 built-in CLI agents** out of the box:
|
||||
ACP supports **15 built-in CLI agents** out of the box:
|
||||
|
||||
| Agent ID | Display Name | Binary | Protocol |
|
||||
| ------------- | ------------------ | ------------- | -------- |
|
||||
@@ -38,12 +38,14 @@ ACP supports **13 built-in CLI agents** out of the box:
|
||||
| `aider` | Aider | `aider` | stdio |
|
||||
| `opencode` | OpenCode | `opencode` | stdio |
|
||||
| `cline` | Cline | `cline` | stdio |
|
||||
| `qwen` | Qwen Code | `qwen --acp` | stdio |
|
||||
| `qwen` | Qwen Code | `qwen --acp` | stdio |
|
||||
| `forge` | ForgeCode | `forge` | stdio |
|
||||
| `amazon-q` | Amazon Q Developer | `q` | stdio |
|
||||
| `interpreter` | Open Interpreter | `interpreter` | stdio |
|
||||
| `cursor-cli` | Cursor CLI | `cursor` | stdio |
|
||||
| `warp` | Warp AI | `warp` | stdio |
|
||||
| `gemini` | Gemini CLI | `gemini` | stdio |
|
||||
| `zcode` | ZCode | `zcode` | stdio |
|
||||
|
||||
### Custom Agents
|
||||
|
||||
@@ -543,7 +545,7 @@ const agents = detectInstalledAgents();
|
||||
## What's Next?
|
||||
|
||||
- **[API Reference](../reference/API_REFERENCE.md)** — REST API endpoints
|
||||
- **[Provider Reference](../reference/PROVIDER_REFERENCE.md)** — All 338 providers
|
||||
- **[Provider Reference](../reference/PROVIDER_REFERENCE.md)** — All 352 providers
|
||||
- **[MCP Server](./MCP-SERVER.md)** — Model Context Protocol integration
|
||||
- **[A2A Server](./A2A-SERVER.md)** — Agent-to-Agent protocol
|
||||
- **[Cloud Agent](./CLOUD_AGENT.md)** — Cloud-based agents
|
||||
|
||||
@@ -90,13 +90,13 @@ The generator preserves content between `<!-- skill:custom-start -->` and `<!--
|
||||
|
||||
## REST API Discovery
|
||||
|
||||
| Endpoint | Method | Description | Auth |
|
||||
| :--------------------------- | :----- | :------------------------------------------------------- | :--------- |
|
||||
| Endpoint | Method | Description | Auth |
|
||||
| :--------------------------- | :----- | :--------------------------------------------------------------- | :--------- |
|
||||
| `/api/agent-skills` | GET | List catalog (optional `?category=api\|cli\|config&area=<area>`) | none |
|
||||
| `/api/agent-skills/{id}` | GET | Get single skill metadata | none |
|
||||
| `/api/agent-skills/{id}/raw` | GET | Fetch SKILL.md as `text/markdown` | none |
|
||||
| `/api/agent-skills/coverage` | GET | Coverage stats (how many SKILL.md files exist) | none |
|
||||
| `/api/agent-skills/generate` | POST | Trigger generator (dryRun/prune/onlyIds) | management |
|
||||
| `/api/agent-skills/{id}` | GET | Get single skill metadata | none |
|
||||
| `/api/agent-skills/{id}/raw` | GET | Fetch SKILL.md as `text/markdown` | none |
|
||||
| `/api/agent-skills/coverage` | GET | Coverage stats (how many SKILL.md files exist) | none |
|
||||
| `/api/agent-skills/generate` | POST | Trigger generator (dryRun/prune/onlyIds) | management |
|
||||
|
||||
Example — list all API skills:
|
||||
|
||||
@@ -150,61 +150,61 @@ See [A2A-SERVER.md](./A2A-SERVER.md) for protocol details.
|
||||
|
||||
### API Skills (23)
|
||||
|
||||
| ID | Area | Entry Point |
|
||||
| :--------------------- | :-------------- | :---------------------------------- |
|
||||
| `omni-auth` | auth | Auth + session management |
|
||||
| `omni-providers` | providers | Provider connection management |
|
||||
| `omni-models` | models | Model catalog and capabilities |
|
||||
| `omni-combos-routing` | combos-routing | Combo routing strategies |
|
||||
| `omni-api-keys` | api-keys | API key management |
|
||||
| `omni-usage-logs` | usage-logs | Usage and cost logs |
|
||||
| `omni-budget` | budget | Budget guards |
|
||||
| `omni-settings` | settings | Global settings |
|
||||
| `omni-proxies` | proxies | Proxy pool management |
|
||||
| `omni-cache` | cache | Semantic + prompt cache |
|
||||
| `omni-compression` | compression | Context compression engines |
|
||||
| `omni-context-rtk` | context-rtk | RTK compression |
|
||||
| `omni-resilience` | resilience | Circuit breakers + cooldowns |
|
||||
| `omni-cli-tools` | cli-tools | CLI tools REST proxy |
|
||||
| `omni-tunnels` | tunnels | Tunnel management |
|
||||
| `omni-sync-cloud` | sync-cloud | Cloud sync |
|
||||
| `omni-db-backups` | db-backups | Database backups |
|
||||
| `omni-webhooks` | webhooks | Webhook event dispatcher |
|
||||
| `omni-mcp` | mcp | MCP server (107 tools, 3 transports) |
|
||||
| `omni-agents-a2a` | agents-a2a | A2A agent protocol |
|
||||
| `omni-version-manager` | version-manager | Version and update management |
|
||||
| `omni-inference` | inference | Direct inference / completions |
|
||||
| ID | Area | Entry Point |
|
||||
| :--------------------- | :-------------- | :----------------------------------- |
|
||||
| `omni-auth` | auth | Auth + session management |
|
||||
| `omni-providers` | providers | Provider connection management |
|
||||
| `omni-models` | models | Model catalog and capabilities |
|
||||
| `omni-combos-routing` | combos-routing | Combo routing strategies |
|
||||
| `omni-api-keys` | api-keys | API key management |
|
||||
| `omni-usage-logs` | usage-logs | Usage and cost logs |
|
||||
| `omni-budget` | budget | Budget guards |
|
||||
| `omni-settings` | settings | Global settings |
|
||||
| `omni-proxies` | proxies | Proxy pool management |
|
||||
| `omni-cache` | cache | Semantic + prompt cache |
|
||||
| `omni-compression` | compression | Context compression engines |
|
||||
| `omni-context-rtk` | context-rtk | RTK compression |
|
||||
| `omni-resilience` | resilience | Circuit breakers + cooldowns |
|
||||
| `omni-cli-tools` | cli-tools | CLI tools REST proxy |
|
||||
| `omni-tunnels` | tunnels | Tunnel management |
|
||||
| `omni-sync-cloud` | sync-cloud | Cloud sync |
|
||||
| `omni-db-backups` | db-backups | Database backups |
|
||||
| `omni-webhooks` | webhooks | Webhook event dispatcher |
|
||||
| `omni-mcp` | mcp | MCP server (110 tools, 3 transports) |
|
||||
| `omni-agents-a2a` | agents-a2a | A2A agent protocol |
|
||||
| `omni-version-manager` | version-manager | Version and update management |
|
||||
| `omni-inference` | inference | Direct inference / completions |
|
||||
|
||||
### CLI Skills (21)
|
||||
|
||||
| ID | Area | CLI Command Root |
|
||||
| :------------------- | :----------------- | :---------------------- |
|
||||
| `cli-serve` | cli-serve | `omniroute serve` |
|
||||
| `cli-health` | cli-health | `omniroute health` |
|
||||
| `cli-providers` | cli-providers | `omniroute providers` |
|
||||
| `cli-keys` | cli-keys | `omniroute keys` |
|
||||
| `cli-models` | cli-models | `omniroute models` |
|
||||
| `cli-chat` | cli-chat | `omniroute chat` |
|
||||
| `cli-routing` | cli-routing | `omniroute routing` |
|
||||
| `cli-resilience` | cli-resilience | `omniroute resilience` |
|
||||
| `cli-compression` | cli-compression | `omniroute compression` |
|
||||
| `cli-contexts` | cli-contexts | `omniroute contexts` |
|
||||
| `cli-cost-usage` | cli-cost-usage | `omniroute cost` |
|
||||
| `cli-mcp` | cli-mcp | `omniroute mcp` |
|
||||
| `cli-a2a` | cli-a2a | `omniroute a2a` |
|
||||
| `cli-tunnel` | cli-tunnel | `omniroute tunnel` |
|
||||
| `cli-backup-sync` | cli-backup-sync | `omniroute backup` |
|
||||
| `cli-policy-audit` | cli-policy-audit | `omniroute policy` |
|
||||
| `cli-batches` | cli-batches | `omniroute batch` |
|
||||
| `cli-eval` | cli-eval | `omniroute eval` |
|
||||
| `cli-plugins-skills` | cli-plugins-skills | `omniroute plugins` |
|
||||
| `cli-setup` | cli-setup | `omniroute setup` |
|
||||
| ID | Area | CLI Command Root |
|
||||
| :-------------------- | :----------------- | :---------------------- |
|
||||
| `cli-serve` | cli-serve | `omniroute serve` |
|
||||
| `cli-health` | cli-health | `omniroute health` |
|
||||
| `cli-providers` | cli-providers | `omniroute providers` |
|
||||
| `cli-keys` | cli-keys | `omniroute keys` |
|
||||
| `cli-models` | cli-models | `omniroute models` |
|
||||
| `cli-chat` | cli-chat | `omniroute chat` |
|
||||
| `cli-routing` | cli-routing | `omniroute routing` |
|
||||
| `cli-resilience` | cli-resilience | `omniroute resilience` |
|
||||
| `cli-compression` | cli-compression | `omniroute compression` |
|
||||
| `cli-contexts` | cli-contexts | `omniroute contexts` |
|
||||
| `cli-cost-usage` | cli-cost-usage | `omniroute cost` |
|
||||
| `cli-mcp` | cli-mcp | `omniroute mcp` |
|
||||
| `cli-a2a` | cli-a2a | `omniroute a2a` |
|
||||
| `cli-tunnel` | cli-tunnel | `omniroute tunnel` |
|
||||
| `cli-backup-sync` | cli-backup-sync | `omniroute backup` |
|
||||
| `cli-policy-audit` | cli-policy-audit | `omniroute policy` |
|
||||
| `cli-batches` | cli-batches | `omniroute batch` |
|
||||
| `cli-eval` | cli-eval | `omniroute eval` |
|
||||
| `cli-plugins-skills` | cli-plugins-skills | `omniroute plugins` |
|
||||
| `cli-setup` | cli-setup | `omniroute setup` |
|
||||
| `cli-skill-collector` | cli-setup | `omniroute skills` |
|
||||
|
||||
### Configuration workflow (1)
|
||||
|
||||
| ID | Area | Entry Point |
|
||||
| :---------------- | :--------------- | :------------------------------- |
|
||||
| ID | Area | Entry Point |
|
||||
| :----------------- | :--------------- | :------------------------------- |
|
||||
| `config-codex-cli` | config-codex-cli | Codex CLI configuration workflow |
|
||||
|
||||
---
|
||||
|
||||
@@ -6,7 +6,7 @@ lastUpdated: 2026-06-28
|
||||
|
||||
# AgentBridge
|
||||
|
||||
AgentBridge is OmniRoute's MITM (Man-in-the-Middle) proxy that intercepts HTTPS traffic from IDE AI agents and reroutes it through OmniRoute's unified routing engine. It supports **9 IDE agents** — Antigravity, Kiro, GitHub Copilot, OpenAI Codex, Cursor, Zed, Claude Code, Open Code, and Trae (investigating) — making OmniRoute the broadest-coverage MITM proxy for AI coding assistants on the market.
|
||||
AgentBridge is OmniRoute's MITM (Man-in-the-Middle) proxy that intercepts HTTPS traffic from IDE AI agents and reroutes it through OmniRoute's unified routing engine. It supports **10 IDE agents** — Antigravity, Kiro, GitHub Copilot, GHE Copilot, OpenAI Codex, Cursor, Zed, Claude Code, Open Code, and Trae (investigating) — making OmniRoute the broadest-coverage MITM proxy for AI coding assistants on the market.
|
||||
|
||||
**Dashboard location:** `/dashboard/tools/agent-bridge`
|
||||
**Sidebar group:** Tools (after Cloud Agents)
|
||||
@@ -22,7 +22,7 @@ When an IDE agent (e.g., GitHub Copilot, Cursor, Claude Code) makes an API call,
|
||||
|
||||
This means you can:
|
||||
|
||||
- **Reroute any agent to any provider**: Copilot talking to OpenAI? Redirect it to Anthropic Claude, Gemini, or any of OmniRoute's 338 providers.
|
||||
- **Reroute any agent to any provider**: Copilot talking to OpenAI? Redirect it to Anthropic Claude, Gemini, or any of OmniRoute's 352 providers.
|
||||
- **Apply model mappings**: `gemini-3-flash` → `claude-sonnet-4.7` transparently at the handler level.
|
||||
- **Observe all agent traffic**: every intercepted request is published to the [Traffic Inspector](./TRAFFIC_INSPECTOR.md).
|
||||
- **Apply OmniRoute resilience**: combo routing, circuit breakers, fallbacks, and cost tracking work for IDE agent traffic too.
|
||||
|
||||
@@ -129,7 +129,7 @@ Body shape for POST (`customAgentBodySchema` in `src/app/api/acp/agents/route.ts
|
||||
|
||||
A uniform interface over third-party cloud coding agents. You submit a prompt + repo URL, OmniRoute dispatches to the right cloud agent, polls status, returns results.
|
||||
|
||||
### Supported agents (3, all confirmed in `src/lib/cloudAgent/agents/`)
|
||||
### Supported agents (4, all confirmed in `src/lib/cloudAgent/agents/`)
|
||||
|
||||
- `codex-cloud` — OpenAI Codex Cloud
|
||||
- `devin` — Cognition Devin
|
||||
@@ -195,7 +195,7 @@ curl -X POST http://localhost:20128/a2a \
|
||||
"method": "message/send",
|
||||
"params": {
|
||||
"messages": [{"role": "user", "content": "Route this prompt"}],
|
||||
"skillId": "smart-routing"
|
||||
"skill": "smart-routing"
|
||||
},
|
||||
"id": 1
|
||||
}'
|
||||
@@ -231,10 +231,13 @@ curl -X POST http://localhost:20128/api/v1/agents/tasks \
|
||||
-H "Cookie: auth_token=..." \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"agentId": "devin",
|
||||
"providerId": "devin",
|
||||
"prompt": "Implement feature X in repo Y",
|
||||
"repo": "https://github.com/user/repo",
|
||||
"branch": "main"
|
||||
"source": {
|
||||
"repoName": "user/repo",
|
||||
"repoUrl": "https://github.com/user/repo",
|
||||
"branch": "main"
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
@@ -269,7 +272,7 @@ curl http://localhost:20128/api/v1/agents/tasks/<task-id> \
|
||||
↓ ↓ └────────────┘
|
||||
External peer Local CLI │
|
||||
agents that binaries on ↓
|
||||
speak A2A v0.3 the host Codex Cloud,
|
||||
speak A2A v0.3 the host Codex Cloud, Cursor,
|
||||
Devin, Jules
|
||||
```
|
||||
|
||||
|
||||
@@ -32,16 +32,17 @@ via API keys), embedded services run on the same machine as OmniRoute and commun
|
||||
|
||||
### Why embedded services?
|
||||
|
||||
Four services are embedded as of v3.8.44:
|
||||
Five services are embedded:
|
||||
|
||||
| Service | npm package | Default port | Purpose |
|
||||
| --------------- | ----------------------------------------------- | :----------: | ------------------------------------------------------------------------------------------------------------------ |
|
||||
| **9Router** | `9router` | 20130 | AI router that OmniRoute can use as a sub-provider. Models exposed as `9router/{sub}/{model}` |
|
||||
| **CLIProxyAPI** | `@anthropic/cli-proxy` (via `cliproxy` binary) | auto | Local proxy adapter for Anthropic CLI auth flows. Provides fallback routing when OAuth tokens expire |
|
||||
| **Mux** | `mux` (headless `mux server`) | 8322 | Local agent-orchestration daemon (coder/mux). Lifecycle-managed only — not a routing target (no LLM proxying). |
|
||||
| **Bifrost** | `@maximhq/bifrost` | 8080 | Go AI-gateway relay backend. When running, auto-selected by the relay route (`/v1/relay/`) |
|
||||
| Service | npm package | Default port | Purpose |
|
||||
| --------------- | ---------------------------------- | :----------: | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **9Router** | `9router` | 20130 | AI router that OmniRoute can use as a sub-provider. Models exposed as `9router/{sub}/{model}` |
|
||||
| **CLIProxyAPI** | GitHub release binary (`cliproxy`) | 8317 | Local proxy adapter for Anthropic CLI auth flows. Provides fallback routing when OAuth tokens expire |
|
||||
| **Mux** | `mux` (headless `mux server`) | 8322 | Local agent-orchestration daemon (coder/mux). Lifecycle-managed only — not a routing target (no LLM proxying). |
|
||||
| **Bifrost** | `@maximhq/bifrost` | 8080 | Go AI-gateway relay backend. When running, auto-selected by the relay route (`/v1/relay/`) |
|
||||
| **Dario** | `@askalf/dario` | 3456 | Claude-subscription proxy — alternative/failover to CLIProxyAPI for Claude-Code-shaped traffic; the injected key becomes `DARIO_ADMIN_TOKEN` gating its `/admin/*` OAuth control plane |
|
||||
|
||||
All four follow the same supervisory model:
|
||||
All five follow the same supervisory model:
|
||||
|
||||
- OmniRoute installs them under `DATA_DIR/services/{name}/` (isolated from OmniRoute's own `package.json`)
|
||||
- OmniRoute spawns and monitors them as child processes
|
||||
@@ -211,7 +212,7 @@ race conditions when, for example, auto-start and a UI button fire simultaneousl
|
||||
All routes under `/api/services/` are **LOCAL_ONLY** (loopback only, hard rule #17).
|
||||
Non-loopback requests receive `403 LOCAL_ONLY` regardless of auth token.
|
||||
|
||||
### 4.1 9Router endpoints (8 routes)
|
||||
### 4.1 9Router endpoints (11 routes)
|
||||
|
||||
#### `POST /api/services/9router/install`
|
||||
|
||||
@@ -429,11 +430,12 @@ SSE stream of live logs from 9Router's stdout/stderr ring buffer.
|
||||
|
||||
---
|
||||
|
||||
### 4.2 CLIProxyAPI endpoints (7 routes)
|
||||
### 4.2 CLIProxyAPI endpoints (10 routes)
|
||||
|
||||
CLIProxyAPI has the same endpoint shape as 9Router minus `rotate-key` (CLIProxyAPI
|
||||
does not require an injected API key; it authenticates via the host's existing CLI
|
||||
config) and `status` includes fewer fields.
|
||||
CLIProxyAPI has the same endpoint shape as 9Router minus `rotate-key`, plus
|
||||
`accounts`, `provider-expose` and `auto-restart-adopted`. It now receives a
|
||||
dedicated data-plane API key injected at spawn (`needsApiKey: true` in
|
||||
`bootstrap.ts`, used for model sync); `status` includes fewer fields.
|
||||
|
||||
| Method | Path | Description |
|
||||
| ------ | ----------------------------------- | ------------------------------------ |
|
||||
@@ -450,7 +452,7 @@ four services using the `[name]` dynamic segment.
|
||||
|
||||
---
|
||||
|
||||
### 4.3 Mux endpoints (7 routes)
|
||||
### 4.3 Mux endpoints (8 routes)
|
||||
|
||||
Mux has the same endpoint shape as CLIProxyAPI — no `rotate-key` route in the API
|
||||
surface (the bearer token is generated the same way as 9Router's via
|
||||
@@ -458,34 +460,34 @@ surface (the bearer token is generated the same way as 9Router's via
|
||||
there is no dedicated rotation endpoint yet). Mux is lifecycle-managed only: unlike
|
||||
9Router, it has no Layer 4 executor and is never registered as a routing provider.
|
||||
|
||||
| Method | Path | Description |
|
||||
| ------ | -------------------------------- | ------------------------------------- |
|
||||
| `POST` | `/api/services/mux/install` | Install Mux from npm (`npm i mux`) |
|
||||
| `POST` | `/api/services/mux/start` | Start Mux (`mux server`) |
|
||||
| `POST` | `/api/services/mux/stop` | Stop Mux |
|
||||
| `POST` | `/api/services/mux/restart` | Restart Mux |
|
||||
| `POST` | `/api/services/mux/update` | Update to newer npm version |
|
||||
| `GET` | `/api/services/mux/status` | Live + DB status |
|
||||
| `POST` | `/api/services/mux/auto-start` | Toggle auto-start |
|
||||
| Method | Path | Description |
|
||||
| ------ | ------------------------------ | ---------------------------------- |
|
||||
| `POST` | `/api/services/mux/install` | Install Mux from npm (`npm i mux`) |
|
||||
| `POST` | `/api/services/mux/start` | Start Mux (`mux server`) |
|
||||
| `POST` | `/api/services/mux/stop` | Stop Mux |
|
||||
| `POST` | `/api/services/mux/restart` | Restart Mux |
|
||||
| `POST` | `/api/services/mux/update` | Update to newer npm version |
|
||||
| `GET` | `/api/services/mux/status` | Live + DB status |
|
||||
| `POST` | `/api/services/mux/auto-start` | Toggle auto-start |
|
||||
|
||||
---
|
||||
|
||||
### 4.4 Bifrost endpoints (7 routes)
|
||||
### 4.4 Bifrost endpoints (8 routes)
|
||||
|
||||
Bifrost is a Go AI-gateway relay backend (`@maximhq/bifrost`). It uses the same
|
||||
endpoint shape as CLIProxyAPI (no `rotate-key` — Bifrost manages its own provider
|
||||
keys in `config.json` under its `-app-dir`).
|
||||
|
||||
| Method | Path | Description |
|
||||
| ------ | ---------------------------------- | ------------------------------------------------------ |
|
||||
| `POST` | `/api/services/bifrost/install` | Install Bifrost from npm (`@maximhq/bifrost`) |
|
||||
| `POST` | `/api/services/bifrost/start` | Start Bifrost on port 8080 (default) |
|
||||
| `POST` | `/api/services/bifrost/stop` | Stop Bifrost |
|
||||
| `POST` | `/api/services/bifrost/restart` | Restart Bifrost |
|
||||
| `POST` | `/api/services/bifrost/update` | Update to newer version |
|
||||
| `GET` | `/api/services/bifrost/status` | Live + DB status |
|
||||
| `POST` | `/api/services/bifrost/auto-start` | Toggle auto-start |
|
||||
| `GET` | `/api/services/bifrost/logs` | SSE log tail (via shared `[name]/logs` dynamic route) |
|
||||
| Method | Path | Description |
|
||||
| ------ | ---------------------------------- | ----------------------------------------------------- |
|
||||
| `POST` | `/api/services/bifrost/install` | Install Bifrost from npm (`@maximhq/bifrost`) |
|
||||
| `POST` | `/api/services/bifrost/start` | Start Bifrost on port 8080 (default) |
|
||||
| `POST` | `/api/services/bifrost/stop` | Stop Bifrost |
|
||||
| `POST` | `/api/services/bifrost/restart` | Restart Bifrost |
|
||||
| `POST` | `/api/services/bifrost/update` | Update to newer version |
|
||||
| `GET` | `/api/services/bifrost/status` | Live + DB status |
|
||||
| `POST` | `/api/services/bifrost/auto-start` | Toggle auto-start |
|
||||
| `GET` | `/api/services/bifrost/logs` | SSE log tail (via shared `[name]/logs` dynamic route) |
|
||||
|
||||
**Routing wiring:** When `BIFROST_BASE_URL` is unset and the supervised Bifrost
|
||||
instance is running, `getBifrostRoutingConfig()` (in `routingBackend.ts`) automatically
|
||||
@@ -494,7 +496,14 @@ always takes precedence.
|
||||
|
||||
---
|
||||
|
||||
### 4.4 Reverse proxy (9Router dashboard embed)
|
||||
### 4.5 Dario endpoints (12 routes)
|
||||
|
||||
Same lifecycle shape as the other services (`install`, `start`, `stop`, `restart`,
|
||||
`update`, `status`, `auto-start`, `auto-restart-adopted`) plus a token-gated OAuth
|
||||
control plane under `admin/`: `admin/accounts`, `admin/import-from-omniroute`,
|
||||
`admin/login-start`, `admin/login-complete` (all behind `DARIO_ADMIN_TOKEN`).
|
||||
|
||||
### 4.6 Reverse proxy (9Router dashboard embed)
|
||||
|
||||
The dashboard embeds the 9Router web UI inside an iframe via an internal reverse
|
||||
proxy at:
|
||||
@@ -552,8 +561,8 @@ OmniRoute:
|
||||
flag, so the token never appears in `ps`/process listings).
|
||||
4. Never returns the plaintext key in any HTTP response.
|
||||
|
||||
CLIProxyAPI does not require an injected key (it authenticates via the host's
|
||||
existing CLI config).
|
||||
CLIProxyAPI receives a dedicated data-plane key injected at spawn
|
||||
(`needsApiKey: true` — used for model sync against the adapter).
|
||||
|
||||
### SSRF defense
|
||||
|
||||
|
||||
@@ -64,7 +64,7 @@ Cursor, Cline, and compatible MCP client setup.
|
||||
|
||||
---
|
||||
|
||||
## Essential Tools (13) — Phase 1
|
||||
## Essential Tools (14) — Phase 1
|
||||
|
||||
| Tool | Scopes | Description |
|
||||
| :------------------------------ | :-------------------- | :----------------------------------------------------------------------------------------------------------------------------- |
|
||||
@@ -215,13 +215,13 @@ curl -X DELETE http://localhost:20128/api/settings/notion
|
||||
|
||||
## Agent Skill Catalog Tools (3)
|
||||
|
||||
Defined in `open-sse/mcp-server/tools/agentSkillTools.ts`. Backed by `src/lib/agentSkills/catalog`. These tools expose the 42-entry Agent Skills documentation catalog to MCP clients and external agents. Scope: `read:catalog`.
|
||||
Defined in `open-sse/mcp-server/tools/agentSkillTools.ts`. Backed by `src/lib/agentSkills/catalog`. These tools expose the 45-entry Agent Skills documentation catalog to MCP clients and external agents. Scope: `read:catalog`.
|
||||
|
||||
| Tool | Scopes | Description |
|
||||
| :-------------------------------- | :------------- | :--------------------------------------------------------------------------------------------------------------- |
|
||||
| `omniroute_agent_skills_list` | `read:catalog` | List all 42 agent skills with optional `category` (api\|cli) and `area` filters; returns metadata + coverage |
|
||||
| `omniroute_agent_skills_get` | `read:catalog` | Get full metadata + SKILL.md content for a single skill by canonical `id` |
|
||||
| `omniroute_agent_skills_coverage` | `read:catalog` | Coverage stats: how many of the 22 API and 20 CLI skills have SKILL.md files on the filesystem vs catalog totals |
|
||||
| Tool | Scopes | Description |
|
||||
| :-------------------------------- | :------------- | :------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `omniroute_agent_skills_list` | `read:catalog` | List all 45 agent skills with optional `category` (api\|cli) and `area` filters; returns metadata + coverage |
|
||||
| `omniroute_agent_skills_get` | `read:catalog` | Get full metadata + SKILL.md content for a single skill by canonical `id` |
|
||||
| `omniroute_agent_skills_coverage` | `read:catalog` | Coverage stats: how many of the 23 API, 21 CLI and 1 config skills have SKILL.md files on the filesystem vs catalog totals |
|
||||
|
||||
See [AGENT-SKILLS.md](./AGENT-SKILLS.md) for the full catalog and how external agents consume it.
|
||||
|
||||
@@ -233,12 +233,12 @@ frameworks ship alongside the MCP server in v3.8.0 and are documented separately
|
||||
|
||||
### Cloud Agents
|
||||
|
||||
Cloud Agents are out-of-process AI coding agents (codex-cloud, devin, jules) wired into
|
||||
Cloud Agents are out-of-process AI coding agents (codex-cloud, cursor-cloud, devin, jules) wired into
|
||||
OmniRoute through the same connection model used for LLM providers. They are exposed via
|
||||
their own REST surface (`/api/v1/agents/*`) and are **not** part of the MCP tool catalog
|
||||
— calling a Cloud Agent does not consume an MCP scope.
|
||||
|
||||
- Implementation: `src/lib/cloudAgent/` (`registry.ts`, `agents/codex-cloud.ts`, `agents/devin.ts`, `agents/jules.ts`).
|
||||
- Implementation: `src/lib/cloudAgent/` (`registry.ts`, `agents/codex.ts`, `agents/cursor.ts`, `agents/devin.ts`, `agents/jules.ts`).
|
||||
- Lifecycle: `createTask`, `getStatus`, `approvePlan`, `sendMessage`, `listSources`.
|
||||
- Documentation: [docs/frameworks/CLOUD_AGENT.md](./CLOUD_AGENT.md).
|
||||
|
||||
@@ -279,32 +279,41 @@ Both SSE and Streamable HTTP transports are blocked until the MCP server is enab
|
||||
MCP tools are authenticated through API key scopes. Scope enforcement is centralized in
|
||||
`open-sse/mcp-server/scopeEnforcement.ts`. Each tool requires specific scopes:
|
||||
|
||||
| Scope | Tools |
|
||||
| :-------------------- | :---------------------------------------------------------------------------------------------------------------- |
|
||||
| `read:health` | `get_health`, `get_provider_metrics`, `simulate_route`, `explain_route`, `best_combo_for_task`, `db_health_check` |
|
||||
| `read:combos` | `list_combos`, `get_combo_metrics`, `simulate_route`, `best_combo_for_task`, `test_combo` |
|
||||
| `write:combos` | `switch_combo`, `set_routing_strategy` |
|
||||
| `read:quota` | `check_quota` |
|
||||
| `read:usage` | `cost_report`, `get_session_snapshot`, `explain_route` |
|
||||
| `read:models` | `list_models_catalog` |
|
||||
| `execute:completions` | `route_request`, `test_combo` |
|
||||
| `execute:search` | `web_search` |
|
||||
| `write:budget` | `set_budget_guard` |
|
||||
| `write:resilience` | `set_resilience_profile`, `db_health_check` |
|
||||
| `pricing:write` | `sync_pricing` |
|
||||
| `read:cache` | `cache_stats` |
|
||||
| `write:cache` | `cache_flush` |
|
||||
| `read:compression` | `compression_status`, `list_compression_combos`, `compression_combo_stats` |
|
||||
| `write:compression` | `compression_configure`, `set_compression_engine` |
|
||||
| `read:proxies` | `oneproxy_fetch`, `oneproxy_rotate`, `oneproxy_stats` |
|
||||
| `read:notion` | `notion_search`, `notion_list_databases`, `notion_get_database`, `notion_query_database`, `notion_read` |
|
||||
| `write:notion` | `notion_append_blocks` |
|
||||
| `read:memory` | `memory_search` |
|
||||
| `write:memory` | `memory_add`, `memory_clear` |
|
||||
| `read:skills` | `skills_list`, `skills_executions` |
|
||||
| `write:skills` | `skills_enable` |
|
||||
| `execute:skills` | `skills_execute` |
|
||||
| `read:catalog` | `agent_skills_list`, `agent_skills_get`, `agent_skills_coverage` |
|
||||
| Scope | Tools |
|
||||
| :-------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `read:health` | `get_health`, `get_provider_metrics`, `simulate_route`, `explain_route`, `best_combo_for_task`, `db_health_check` |
|
||||
| `read:combos` | `list_combos`, `get_combo_metrics`, `simulate_route`, `best_combo_for_task`, `test_combo` |
|
||||
| `write:combos` | `switch_combo`, `set_routing_strategy` |
|
||||
| `read:quota` | `check_quota` |
|
||||
| `read:usage` | `cost_report`, `get_session_snapshot`, `explain_route` |
|
||||
| `read:models` | `list_models_catalog` |
|
||||
| `execute:completions` | `route_request`, `test_combo` |
|
||||
| `execute:search` | `web_search`, `x_search`, `web_fetch` |
|
||||
| `write:budget` | `set_budget_guard` |
|
||||
| `write:resilience` | `set_resilience_profile`, `db_health_check` |
|
||||
| `pricing:write` | `sync_pricing` |
|
||||
| `read:cache` | `cache_stats` |
|
||||
| `write:cache` | `cache_flush` |
|
||||
| `read:compression` | `compression_status`, `list_compression_combos`, `compression_combo_stats` |
|
||||
| `write:compression` | `compression_configure`, `set_compression_engine` |
|
||||
| `read:proxies` | `oneproxy_fetch`, `oneproxy_rotate`, `oneproxy_stats` |
|
||||
| `read:notion` | `notion_search`, `notion_get_page`, `notion_list_block_children`, `notion_query_database`, `notion_get_database` |
|
||||
| `write:notion` | `notion_append_blocks` |
|
||||
| `read:memory` | `memory_search` |
|
||||
| `write:memory` | `memory_add`, `memory_clear` |
|
||||
| `read:skills` | `skills_list`, `skills_executions` |
|
||||
| `write:skills` | `skills_enable` |
|
||||
| `execute:skills` | `skills_execute` |
|
||||
| `read:catalog` | `agent_skills_list`, `agent_skills_get`, `agent_skills_coverage` |
|
||||
| `read:tools` | `omniroute_tool_search` |
|
||||
| `read:radar` | `omniroute_radar_catalog` |
|
||||
| `read:gamification` | `gamification_profile`, `gamification_rank`, `gamification_leaderboard`, `gamification_badges`, `gamification_servers`, `gamification_anomalies` |
|
||||
| `write:gamification` | `gamification_invite`, `gamification_transfer` |
|
||||
| `read:plugins` | `plugin_list`, `plugin_executions` |
|
||||
| `write:plugins` | `plugin_scan`, `plugin_install`, `plugin_uninstall`, `plugin_activate`, `plugin_deactivate`, `plugin_configure` |
|
||||
| `read:obsidian` | 13 read tools — `obsidian_list_vault`, `obsidian_read_note`, `obsidian_search_simple`, `obsidian_search_structured`, `obsidian_get_periodic_note`, `obsidian_sync_status`, … |
|
||||
| `write:obsidian` | 9 write tools — `obsidian_write_note`, `obsidian_append_note`, `obsidian_patch_note`, `obsidian_move_note`, `obsidian_delete_note`, `obsidian_sync_trigger`, … |
|
||||
| `read:local-corpus` | `local_corpus_search`, `local_corpus_read`, `local_corpus_status` |
|
||||
|
||||
Wildcard scopes are supported: `read:*` grants all read-scopes, `*` grants full access.
|
||||
|
||||
@@ -409,7 +418,7 @@ The heartbeat snapshot contains:
|
||||
"transport": "stdio",
|
||||
"scopesEnforced": false,
|
||||
"allowedScopes": [],
|
||||
"toolCount": 43
|
||||
"toolCount": 110
|
||||
}
|
||||
```
|
||||
|
||||
@@ -438,7 +447,7 @@ Use the dashboard or the `/api/mcp/audit` and `/api/mcp/audit/stats` REST endpoi
|
||||
| `open-sse/mcp-server/audit.ts` | Tool call audit logging (`mcp_tool_audit`) |
|
||||
| `open-sse/mcp-server/runtimeHeartbeat.ts` | stdio heartbeat writer (`mcp-heartbeat.json`) |
|
||||
| `open-sse/mcp-server/descriptionCompressor.ts` | Description compression for tool / prompt / resource registries |
|
||||
| `open-sse/mcp-server/schemas/tools.ts` | Zod schemas + tool registry (`MCP_TOOLS`, 34 entries) |
|
||||
| `open-sse/mcp-server/schemas/tools.ts` | Zod schemas + tool registry (`MCP_TOOLS`, 45 entries) |
|
||||
| `open-sse/mcp-server/tools/advancedTools.ts` | Phase 2 + cache + 1proxy tool handlers |
|
||||
| `open-sse/mcp-server/tools/compressionTools.ts` | Compression tool handlers |
|
||||
| `open-sse/mcp-server/tools/memoryTools.ts` | Memory tool definitions (3 tools) |
|
||||
|
||||
@@ -152,7 +152,7 @@ amortizes the backfill cost across real requests without blocking startup.
|
||||
`limit` pending entries per request. Progress can be polled via
|
||||
`GET /api/memory/engine-status` (`vectorStore.needsReindex`).
|
||||
|
||||
The `memory_vec_meta` table (migration `073_memory_vec.sql`) stores:
|
||||
The `memory_vec_meta` table (migration `083_memory_vec.sql`) stores:
|
||||
|
||||
- `active_dim` — current vector dimension (null = not yet calibrated).
|
||||
- `embedding_signature` — `${source}:${model}:${dim}` used to detect changes.
|
||||
@@ -568,7 +568,7 @@ the legacy/global settings surface.
|
||||
## Caching
|
||||
|
||||
`src/lib/memory/store.ts` keeps an in-process LRU-ish cache
|
||||
(`MEMORY_CACHE_TTL = 5 min`, `MEMORY_MAX_CACHE_SIZE = 10 000`, with 20 %
|
||||
(`MEMORY_CACHE_TTL = 1 min`, `MEMORY_MAX_CACHE_SIZE = 500`, with 20 %
|
||||
oldest eviction) for `getMemory(id)` reads, plus a generic key/value
|
||||
`memoryCache` layer (`src/lib/memory/cache.ts`) with `get`/`set`/`invalidate`
|
||||
methods used by callers that want their own scoped cache (1 000-entry LRU,
|
||||
@@ -608,7 +608,7 @@ default TTL 5 min).
|
||||
- `src/lib/db/memoryVec.ts` — CRUD for `memory_vec_meta`
|
||||
- `src/lib/db/migrations/015_create_memories.sql`,
|
||||
`022_add_memory_fts5.sql`, `023_fix_memory_fts_uuid.sql`,
|
||||
`073_memory_vec.sql`
|
||||
`083_memory_vec.sql`
|
||||
- `src/app/api/memory/route.ts`, `[id]/route.ts`, `health/route.ts`
|
||||
- `src/app/api/memory/retrieve-preview/route.ts`
|
||||
- `src/app/api/memory/engine-status/route.ts`
|
||||
@@ -628,14 +628,15 @@ default TTL 5 min).
|
||||
|
||||
OmniRoute's memory engine supports **four embedding sources** (`src/lib/memory/embedding/`). Each has different trade-offs in **latency, cost, model quality, and setup complexity**.
|
||||
|
||||
### The Four Providers
|
||||
### The Embedding Sources
|
||||
|
||||
| Provider | Source | Latency | Cost | Quality | Setup |
|
||||
| -------------- | ------------------------------------------ | ------------------------------- | -------------------- | -------------------------- | ------------------ |
|
||||
| `transformers` | Local ONNX model (Xenova/all-MiniLM-L6-v2) | ~50-150ms (CPU) | Free | Good | `npm install` only |
|
||||
| `static` | Pre-computed vectors (cached) | <1ms | Free | N/A (depends on cache hit) | None |
|
||||
| `remote` | OpenAI / Cohere / Voyage API | ~100-300ms | $0.02-0.10/1M tokens | Excellent | API key |
|
||||
| `cache` | In-memory LRU layer over any source | <1ms (hit), full latency (miss) | Free | Same as underlying | None |
|
||||
| Provider | Source | Latency | Cost | Quality | Setup |
|
||||
| -------------- | ------------------------------------------ | ------------------------------- | -------------------- | -------------------------- | ----------------------------------- |
|
||||
| `transformers` | Local ONNX model (Xenova/all-MiniLM-L6-v2) | ~50-150ms (CPU) | Free | Good | `npm install` only |
|
||||
| `static` | Pre-computed vectors (cached) | <1ms | Free | N/A (depends on cache hit) | None |
|
||||
| `remote` | OpenAI / Cohere / Voyage API | ~100-300ms | $0.02-0.10/1M tokens | Excellent | API key |
|
||||
| `auto` | Picks the best available source at runtime | Same as chosen source | Free | Same as chosen source | None |
|
||||
| _(cache)_ | In-memory LRU layer over any source | <1ms (hit), full latency (miss) | Free | Same as underlying | Always on (not a selectable source) |
|
||||
|
||||
### Decision Tree
|
||||
|
||||
@@ -1092,7 +1093,7 @@ memoryManager.register(brainBackend);
|
||||
npx vitest run src/lib/memory/__tests__/generic-backend.test.ts --reporter=verbose
|
||||
```
|
||||
|
||||
Expected output: **26 tests, all passing** covering:
|
||||
Expected output: **35 tests, all passing** covering:
|
||||
|
||||
- Constructor (2)
|
||||
- Health check (4) — success, failure 500, network error, latency
|
||||
|
||||
@@ -6,9 +6,9 @@ lastUpdated: 2026-06-28
|
||||
|
||||
# open-sse Architecture
|
||||
|
||||
> **TL;DR**: `open-sse/` is the core streaming engine that powers every LLM request in OmniRoute. It contains ~900 files implementing the request pipeline, executors, services, MCP server, and translation layer. This guide explains how the pieces fit together.
|
||||
> **TL;DR**: `open-sse/` is the core streaming engine that powers every LLM request in OmniRoute. It contains ~1,400 files implementing the request pipeline, executors, services, MCP server, and translation layer. This guide explains how the pieces fit together.
|
||||
|
||||
**Source:** `open-sse/` (workspace package, ~900 files; 811 `.ts`)
|
||||
**Source:** `open-sse/` (workspace package, ~1,440 non-test `.ts` files)
|
||||
|
||||
---
|
||||
|
||||
@@ -49,13 +49,13 @@ open-sse/
|
||||
### Module Counts
|
||||
|
||||
| Directory | Files | Purpose |
|
||||
| `executors/` | 68 | Per-provider HTTP executors (unified via DefaultExecutor factory) |
|
||||
| `handlers/` | 16 | Request entry points (chatCore, responses, embeddings) |
|
||||
| `services/` | ~298 | Routing, caching, rate limiting, refresh, etc. |
|
||||
| `translator/` | ~27 | Format conversion (OpenAI ↔ Claude ↔ Gemini) |
|
||||
| `mcp-server/` | 32 | MCP tools and transports |
|
||||
| `utils/` | ~65 | Cross-cutting utilities (logging, error, stream) |
|
||||
| `config/` | ~10 | Provider configs, constants, registries |
|
||||
| `executors/` | 167 | Per-provider HTTP executors (unified via DefaultExecutor factory) |
|
||||
| `handlers/` | 157 | Request entry points (chatCore, responses, embeddings) |
|
||||
| `services/` | ~536 | Routing, caching, rate limiting, refresh, etc. |
|
||||
| `translator/` | 56 | Format conversion (OpenAI ↔ Claude ↔ Gemini) |
|
||||
| `mcp-server/` | 44 | MCP tools and transports |
|
||||
| `utils/` | ~108 | Cross-cutting utilities (logging, error, stream) |
|
||||
| `config/` | ~339 | Provider configs, constants, registries |
|
||||
|
||||
---
|
||||
|
||||
@@ -105,7 +105,7 @@ Resolves the request to a concrete `(provider, model, account, credentials)` tup
|
||||
|
||||
For `auto/*` models, this stage also:
|
||||
|
||||
- Runs the **9-factor scoring** algorithm (`services/autoCombo/`)
|
||||
- Runs the **15-factor scoring** algorithm (`services/autoCombo/`)
|
||||
- Selects a `provider+model` pair based on health, cost, latency, etc.
|
||||
|
||||
### Stage 2: Translate (translator/)
|
||||
@@ -245,7 +245,7 @@ Supports **19 routing strategies** (see `src/shared/constants/routingStrategies.
|
||||
| `reset-window` | Reset window-based routing |
|
||||
| `headroom` | Most remaining quota headroom first |
|
||||
| `strict-random` | Truly uniform (no quality weighting) |
|
||||
| `auto` | Use 9-factor scoring (`autoCombo/`) |
|
||||
| `auto` | Use 15-factor scoring (`autoCombo/`) |
|
||||
| `lkgp` | Last known good provider first |
|
||||
| `context-optimized` | Best for long-context requests |
|
||||
| `fusion` | Fan out to a panel in parallel, then synthesize via a judge (`fusion.ts`) |
|
||||
@@ -280,7 +280,7 @@ Services are **focused, single-purpose modules** that handlers compose. The big
|
||||
### Routing & Combo
|
||||
|
||||
- `combo.ts` — entry point for combo-routed requests
|
||||
- `services/autoCombo/` — 9-factor scoring, 8 auto routing strategies
|
||||
- `services/autoCombo/` — 15-factor scoring, 8 auto routing strategies
|
||||
- `wildcardRouter.ts` — matches wildcard routes (`gpt-*`)
|
||||
- `modelFamilyFallback.ts` — T5 intra-family fallback
|
||||
|
||||
@@ -406,9 +406,9 @@ Common translations:
|
||||
|
||||
`open-sse/mcp-server/` implements the **Model Context Protocol** server:
|
||||
|
||||
- **30+ tools** (provider management, combos, memory, cache, compression, 1proxy, skills)
|
||||
- **110 tools** (provider management, combos, memory, cache, compression, proxy, skills, gamification, plugins, Notion, Obsidian, local corpus)
|
||||
- **3 transports**: stdio, SSE, Streamable HTTP
|
||||
- **31 scopes** for fine-grained authorization
|
||||
- **33 scopes** for fine-grained authorization
|
||||
|
||||
### Tool Registration
|
||||
|
||||
@@ -485,13 +485,13 @@ This handles:
|
||||
|
||||
`open-sse/config/` holds the configuration layer:
|
||||
|
||||
| File | Purpose |
|
||||
| ----------------------------- | --------------------------------- |
|
||||
| `providerRegistry.ts` | 338 provider definitions |
|
||||
| `providerModels.ts` | Model aliases, format mapping |
|
||||
| `constants.ts` | Timeouts, limits, status codes |
|
||||
| `defaultThinkingSignature.ts` | Default Claude thinking signature |
|
||||
| `modelStrip.ts` (in services) | Per-provider field stripping |
|
||||
| File | Purpose |
|
||||
| ----------------------------- | ------------------------------------------------- |
|
||||
| `providerRegistry.ts` | Chat-model registry over the 352-provider catalog |
|
||||
| `providerModels.ts` | Model aliases, format mapping |
|
||||
| `constants.ts` | Timeouts, limits, status codes |
|
||||
| `defaultThinkingSignature.ts` | Default Claude thinking signature |
|
||||
| `modelStrip.ts` (in services) | Per-provider field stripping |
|
||||
|
||||
### Provider Registry Schema
|
||||
|
||||
@@ -570,7 +570,7 @@ The routing engine has strict performance budgets:
|
||||
- [ARCHITECTURE.md](../architecture/ARCHITECTURE.md) — high-level architecture
|
||||
- [CODEBASE_DOCUMENTATION.md](../architecture/CODEBASE_DOCUMENTATION.md) — engineering reference
|
||||
- [REPOSITORY_MAP.md](../architecture/REPOSITORY_MAP.md) — directory-by-directory
|
||||
- [AUTO-COMBO.md](../routing/AUTO-COMBO.md) — 9-factor scoring
|
||||
- [AUTO-COMBO.md](../routing/AUTO-COMBO.md) — 15-factor scoring
|
||||
- [MCP-SERVER.md](./MCP-SERVER.md) — MCP server
|
||||
- [A2A-SERVER.md](./A2A-SERVER.md) — A2A server
|
||||
- Source: `open-sse/` (400+ files, ~143K LOC)
|
||||
|
||||
@@ -18,17 +18,14 @@ keep failing.
|
||||
|
||||
## Supported Events
|
||||
|
||||
The `WebhookEvent` type (`src/lib/webhookDispatcher.ts`) currently models:
|
||||
The `WebhookEvent` type (`src/lib/webhooks/eventDescriptions.ts`, consumed by `src/lib/webhookDispatcher.ts`) currently models exactly four events:
|
||||
|
||||
| Event | Fires when |
|
||||
| -------------------- | --------------------------------------------------------- |
|
||||
| `request.completed` | A proxied request completes successfully |
|
||||
| `request.failed` | A proxied request fails after all retries/fallback |
|
||||
| `provider.error` | A provider returns an error eligible for circuit-breaking |
|
||||
| `provider.recovered` | A previously failing provider returns to a healthy state |
|
||||
| `quota.exceeded` | An API key crosses a budget/quota threshold |
|
||||
| `combo.switched` | A combo strategy switches its primary target |
|
||||
| `test.ping` | Synthetic event used by the test endpoint |
|
||||
| Event | Fires when |
|
||||
| ------------------- | -------------------------------------------------- |
|
||||
| `request.completed` | A proxied request completes successfully |
|
||||
| `request.failed` | A proxied request fails after all retries/fallback |
|
||||
| `quota.exceeded` | An API key crosses a budget/quota threshold |
|
||||
| `test.ping` | Synthetic event used by the test endpoint |
|
||||
|
||||
Subscriptions accept the literal `"*"` to receive every event. Unknown event
|
||||
names in `events` are ignored at dispatch time.
|
||||
@@ -123,23 +120,26 @@ Table `webhooks` (migration `011_webhooks.sql`):
|
||||
| `last_status` | INT | HTTP status of the last attempt (0 = network) |
|
||||
| `failure_count` | INT | Resets to 0 on success, +1 on failure |
|
||||
|
||||
There is **no separate `webhook_deliveries` table** in the current schema —
|
||||
delivery history is aggregated on the `webhooks` row. If you need full audit
|
||||
history, consume `request.completed` / `audit` style events from a downstream
|
||||
log store.
|
||||
Delivery history is persisted in the dedicated `webhook_deliveries` table
|
||||
(migration `069_webhook_deliveries.sql`, written via
|
||||
`src/lib/db/webhookDeliveries.ts::insertDelivery` on every attempt), in addition
|
||||
to the aggregate counters on the `webhooks` row. Kind metadata (Slack / Discord /
|
||||
Telegram / custom payload transformers) was added by `070_webhooks_kind_metadata.sql`.
|
||||
|
||||
## REST API
|
||||
|
||||
All endpoints require management auth (`requireManagementAuth`).
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
| ------------------------- | ------ | ------------------------------- |
|
||||
| `/api/webhooks` | GET | List webhooks (secrets masked) |
|
||||
| `/api/webhooks` | POST | Create webhook |
|
||||
| `/api/webhooks/[id]` | GET | Webhook detail (full secret) |
|
||||
| `/api/webhooks/[id]` | PUT | Update fields |
|
||||
| `/api/webhooks/[id]` | DELETE | Remove |
|
||||
| `/api/webhooks/[id]/test` | POST | Fire a `test.ping` (no retries) |
|
||||
| Endpoint | Method | Description |
|
||||
| ------------------------------- | ------ | ---------------------------------------- |
|
||||
| `/api/webhooks` | GET | List webhooks (secrets masked) |
|
||||
| `/api/webhooks` | POST | Create webhook |
|
||||
| `/api/webhooks/[id]` | GET | Webhook detail (full secret) |
|
||||
| `/api/webhooks/[id]` | PUT | Update fields |
|
||||
| `/api/webhooks/[id]` | DELETE | Remove |
|
||||
| `/api/webhooks/[id]/test` | POST | Fire a `test.ping` (no retries) |
|
||||
| `/api/webhooks/[id]/deliveries` | GET | Recent delivery attempts for one webhook |
|
||||
| `/api/webhooks/validate-url` | POST | Pre-flight URL validation (SSRF guard) |
|
||||
|
||||
`GET /api/webhooks` masks the secret to `<first 10 chars>...` to avoid leaking
|
||||
on listing pages. Use the `[id]` GET when you actually need the secret.
|
||||
@@ -153,7 +153,7 @@ curl -X POST http://localhost:20128/api/webhooks \
|
||||
-d '{
|
||||
"url": "https://hooks.slack.com/services/...",
|
||||
"secret": "whsec_my_shared_secret",
|
||||
"events": ["quota.exceeded", "provider.error"],
|
||||
"events": ["quota.exceeded", "request.failed"],
|
||||
"description": "Slack alerts"
|
||||
}'
|
||||
```
|
||||
@@ -203,21 +203,6 @@ The dashboard page at `/dashboard/webhooks` (see
|
||||
}
|
||||
```
|
||||
|
||||
### provider.error
|
||||
|
||||
```json
|
||||
{
|
||||
"event": "provider.error",
|
||||
"timestamp": "2026-05-13T20:31:00.000Z",
|
||||
"data": {
|
||||
"provider": "anthropic",
|
||||
"status": 503,
|
||||
"consecutive_failures": 5,
|
||||
"circuit_state": "open"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### test.ping
|
||||
|
||||
```json
|
||||
@@ -255,5 +240,5 @@ absence).
|
||||
|
||||
- [API_REFERENCE.md](../reference/API_REFERENCE.md) — full management API surface
|
||||
- [RESILIENCE_GUIDE.md](../architecture/RESILIENCE_GUIDE.md) — circuit breaker / cooldown
|
||||
semantics that drive `provider.error` / `provider.recovered`
|
||||
semantics behind provider failures surfaced via `request.failed`
|
||||
- Source: `src/lib/webhookDispatcher.ts`, `src/lib/db/webhooks.ts`
|
||||
|
||||
@@ -25,6 +25,7 @@
|
||||
"SEARCH_TOOLS_STUDIO",
|
||||
"SKILLS",
|
||||
"TRAFFIC_INSPECTOR",
|
||||
"WEBHOOKS"
|
||||
"WEBHOOKS",
|
||||
"LOG-EXPORT"
|
||||
]
|
||||
}
|
||||
|
||||
@@ -41,9 +41,9 @@ Confirmed from `electron/package.json`:
|
||||
|
||||
| Package | Version |
|
||||
| ------------------ | --------------------------------------------------------- |
|
||||
| `electron` | `^41.5.1` |
|
||||
| `electron-builder` | `^26.10.0` |
|
||||
| `electron-updater` | `^6.8.5` |
|
||||
| `electron` | `^43.4.1` |
|
||||
| `electron-builder` | `^26.15.3` |
|
||||
| `electron-updater` | `^6.8.9` |
|
||||
| `better-sqlite3` | root `^13.0.2` (Node-API prebuilds — no Electron rebuild) |
|
||||
| App version | `3.8.0` |
|
||||
| App id | `online.omniroute.desktop` |
|
||||
|
||||
@@ -18,7 +18,7 @@ Visual guide to every section of the OmniRoute dashboard.
|
||||
|
||||
The v3.7.x → v3.8.0 cycle added zero-config auto routing, new providers, OAuth flows, deeper resilience, and a much richer CLI experience. Headline features below — full details further in the document and in linked specs.
|
||||
|
||||
- 🤖 **Auto Combo / Zero-config auto-routing** — use prefixes `auto/coding`, `auto/fast`, `auto/cheap`, `auto/offline`, `auto/smart`, `auto/lkgp`. Backed by a 15-factor scoring engine and 4 curated **mode packs** (ship-fast, cost-saver, quality-first, offline-friendly)
|
||||
- 🤖 **Auto Combo / Zero-config auto-routing** — use prefixes `auto/coding`, `auto/fast`, `auto/cheap`, `auto/offline`, `auto/smart`, `auto/lkgp`, `auto/chaos`. Backed by a 15-factor scoring engine and 6 curated **mode packs** (ship-fast, cost-saver, quality-first, offline-friendly)
|
||||
- 🆕 **Command Code provider** (#2199) — first-class registration with model catalog and quota tracking
|
||||
- 🆕 **Z.AI provider** — new free-tier provider with quota labels
|
||||
- 🎬 **KIE media expansion** — extended catalog including video generation models
|
||||
|
||||
@@ -280,7 +280,7 @@ omniroute config lang set pt-BR
|
||||
# View current language
|
||||
omniroute config lang get
|
||||
|
||||
# List all 42 available languages
|
||||
# List all 43 available languages
|
||||
omniroute config lang list
|
||||
|
||||
# JSON output
|
||||
|
||||
@@ -247,7 +247,7 @@ Add to your MCP settings:
|
||||
}
|
||||
```
|
||||
|
||||
**Full MCP documentation:** [MCP Server README](../../open-sse/mcp-server/README.md) — 107 tools, IDE configs, Python/TS/Go clients.
|
||||
**Full MCP documentation:** [MCP Server README](../../open-sse/mcp-server/README.md) — 110 tools, IDE configs, Python/TS/Go clients.
|
||||
|
||||
### A2A Setup (Agent-to-Agent Protocol)
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ lastUpdated: 2026-06-28
|
||||
|
||||
# OmniRoute Tiers — User Guide
|
||||
|
||||
OmniRoute organizes the 207+ supported providers into 3 economic tiers. Each
|
||||
OmniRoute organizes the 352 supported providers into 3 economic tiers. Each
|
||||
request travels through them in order until one returns successfully — you
|
||||
get the cheapest viable response without ever writing fallback code.
|
||||
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 352 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
@@ -13,11 +12,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -62,7 +61,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -107,7 +106,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -129,7 +128,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -170,7 +169,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -187,7 +186,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -200,7 +199,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -220,7 +219,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -228,7 +227,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -251,13 +250,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -268,7 +267,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -284,19 +283,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -314,7 +313,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -346,35 +345,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -389,13 +387,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -439,7 +437,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -447,9 +445,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -467,9 +465,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -480,11 +478,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
4833
docs/openapi.yaml
4833
docs/openapi.yaml
File diff suppressed because it is too large
Load Diff
@@ -12,7 +12,7 @@ lastUpdated: 2026-08-23
|
||||
|
||||
- `src/lib/db/core.ts` — singleton + SCHEMA_SQL (17 base tables)
|
||||
- `src/lib/db/migrationRunner.ts` — versioned migrations
|
||||
- `src/lib/db/migrations/` — 106 versioned SQL files
|
||||
- `src/lib/db/migrations/` — 167 versioned SQL files
|
||||
- `src/lib/db/encryption.ts` — encryption helpers
|
||||
- `src/lib/db/backup.ts` — backup export/import
|
||||
- `src/lib/db/healthCheck.ts` — health diagnostics
|
||||
|
||||
@@ -1,14 +1,14 @@
|
||||
---
|
||||
title: "API Reference"
|
||||
version: 3.8.50
|
||||
lastUpdated: 2026-08-18
|
||||
version: 3.8.51
|
||||
lastUpdated: 2026-08-31
|
||||
---
|
||||
|
||||
# API Reference
|
||||
|
||||
🌐 **Languages:** 🇺🇸 [English](./API_REFERENCE.md) | 🇧🇷 [Português (Brasil)](../i18n/pt-BR/docs/reference/API_REFERENCE.md) | 🇪🇸 [Español](../i18n/es/docs/reference/API_REFERENCE.md) | 🇫🇷 [Français](../i18n/fr/docs/reference/API_REFERENCE.md) | 🇮🇹 [Italiano](../i18n/it/docs/reference/API_REFERENCE.md) | 🇷🇺 [Русский](../i18n/ru/docs/reference/API_REFERENCE.md) | 🇨🇳 [中文 (简体)](../i18n/zh-CN/docs/reference/API_REFERENCE.md) | 🇩🇪 [Deutsch](../i18n/de/docs/reference/API_REFERENCE.md) | 🇮🇳 [हिन्दी](../i18n/in/docs/reference/API_REFERENCE.md) | 🇹🇭 [ไทย](../i18n/th/docs/reference/API_REFERENCE.md) | 🇺🇦 [Українська](../i18n/uk-UA/docs/reference/API_REFERENCE.md) | 🇸🇦 [العربية](../i18n/ar/docs/reference/API_REFERENCE.md) | 🇯🇵 [日本語](../i18n/ja/docs/reference/API_REFERENCE.md) | 🇻🇳 [Tiếng Việt](../i18n/vi/docs/reference/API_REFERENCE.md) | 🇧🇬 [Български](../i18n/bg/docs/reference/API_REFERENCE.md) | 🇩🇰 [Dansk](../i18n/da/docs/reference/API_REFERENCE.md) | 🇫🇮 [Suomi](../i18n/fi/docs/reference/API_REFERENCE.md) | 🇮🇱 [עברית](../i18n/he/docs/reference/API_REFERENCE.md) | 🇭🇺 [Magyar](../i18n/hu/docs/reference/API_REFERENCE.md) | 🇮🇩 [Bahasa Indonesia](../i18n/id/docs/reference/API_REFERENCE.md) | 🇰🇷 [한국어](../i18n/ko/docs/reference/API_REFERENCE.md) | 🇲🇾 [Bahasa Melayu](../i18n/ms/docs/reference/API_REFERENCE.md) | 🇳🇱 [Nederlands](../i18n/nl/docs/reference/API_REFERENCE.md) | 🇳🇴 [Norsk](../i18n/no/docs/reference/API_REFERENCE.md) | 🇵🇹 [Português (Portugal)](../i18n/pt/docs/reference/API_REFERENCE.md) | 🇷🇴 [Română](../i18n/ro/docs/reference/API_REFERENCE.md) | 🇵🇱 [Polski](../i18n/pl/docs/reference/API_REFERENCE.md) | 🇸🇰 [Slovenčina](../i18n/sk/docs/reference/API_REFERENCE.md) | 🇸🇪 [Svenska](../i18n/sv/docs/reference/API_REFERENCE.md) | 🇵🇭 [Filipino](../i18n/phi/docs/reference/API_REFERENCE.md) | 🇨🇿 [Čeština](../i18n/cs/docs/reference/API_REFERENCE.md)
|
||||
|
||||
Complete reference for all OmniRoute API endpoints.
|
||||
Core reference for the OmniRoute API. It covers the public `/v1` surface and the most-used management endpoints; the machine-readable [`docs/openapi.yaml`](../openapi.yaml) and the route tree under `src/app/api/` are the exhaustive sources.
|
||||
|
||||
---
|
||||
|
||||
@@ -1613,11 +1613,11 @@ devin-cli, etc.). See [Provider Reference](./PROVIDER_REFERENCE.md) for the full
|
||||
| Method | Path | Description |
|
||||
| ------ | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| GET | `/api/cli-tools/all-statuses` | Status of all CLI tools (installed, version, last seen) |
|
||||
| GET | `/api/cli-tools/[id]/status` | Status of a specific CLI tool (id can be: antigravity, chipotle, commandCode, devin-cli, etc.) |
|
||||
| GET | `/api/cli-tools/status` | Status detail for one CLI tool (`?tool=` query) |
|
||||
| POST | `/api/cli-tools/apply` | Write a tool's generated config (`dryRun` previews; `422` + `containerEphemeralTarget` when containerized; `migration` notes a legacy Codex YAML) |
|
||||
| GET | `/api/cli-tools/backups` | List CLI tool configuration backups |
|
||||
| POST | `/api/cli-tools/backups` | Create a backup of all CLI tool configurations |
|
||||
| POST | `/api/cli-tools/[id]/restore` | Restore a CLI tool from a backup |
|
||||
| POST | `/api/cli-tools/backups` | Restore: same endpoint with `{tool, backupId}` in the body restores that backup |
|
||||
| GET | `/api/cli-tools/antigravity-mitm` | Antigravity MITM proxy status (the "antigravity-mitm" CLI tool) |
|
||||
| POST | `/api/cli-tools/antigravity-mitm/alias` | Configure antigravity-mitm aliases |
|
||||
|
||||
@@ -1664,16 +1664,14 @@ Manage the semantic cache and reasoning cache.
|
||||
|
||||
Manage persistent memory (FTS5 + vector embeddings).
|
||||
|
||||
| Method | Path | Description |
|
||||
| ------ | -------------------- | --------------------------------------------------------------------- |
|
||||
| GET | `/api/memory` | List memory entries (filter by scope, type, search query) |
|
||||
| POST | `/api/memory` | Create a new memory entry — body: `{scope, type, content, metadata?}` |
|
||||
| GET | `/api/memory/[id]` | Get a specific memory entry |
|
||||
| PUT | `/api/memory/[id]` | Update a memory entry |
|
||||
| DELETE | `/api/memory/[id]` | Delete a memory entry |
|
||||
| GET | `/api/memory/search` | Search memory (FTS5 + vector) |
|
||||
| POST | `/api/memory/clear` | Clear memory entries (with filters) |
|
||||
| GET | `/api/memory/stats` | Memory statistics (total entries, embedding coverage, etc.) |
|
||||
| Method | Path | Description |
|
||||
| ------ | ------------------ | ----------------------------------------------------------------------- |
|
||||
| GET | `/api/memory` | List memory entries (filter by scope, type, search query) |
|
||||
| POST | `/api/memory` | Create a new memory entry — body: `{scope, type, content, metadata?}` |
|
||||
| GET | `/api/memory/[id]` | Get a specific memory entry |
|
||||
| PUT | `/api/memory/[id]` | Update a memory entry |
|
||||
| DELETE | `/api/memory/[id]` | Delete a memory entry |
|
||||
| GET | `/api/memory?q=` | Search memory (FTS5 + vector) — stats are included in the same response |
|
||||
|
||||
**Auth:** Requires management session or management-scoped API key.
|
||||
|
||||
@@ -1690,7 +1688,6 @@ Manage webhook subscriptions for events.
|
||||
| GET | `/api/webhooks/[id]` | Get a specific webhook subscription |
|
||||
| PUT | `/api/webhooks/[id]` | Update a webhook subscription |
|
||||
| DELETE | `/api/webhooks/[id]` | Delete a webhook subscription |
|
||||
| GET | `/api/webhooks/events` | List all available webhook event types |
|
||||
| GET | `/api/webhooks/[id]/deliveries` | List delivery history for a webhook (success/failure log) |
|
||||
| POST | `/api/webhooks/[id]/test` | Send a test event to a webhook |
|
||||
|
||||
@@ -1723,15 +1720,15 @@ See [Skills Framework](../frameworks/SKILLS.md) for full details.
|
||||
|
||||
Manage OmniRoute plugins (third-party extensions).
|
||||
|
||||
| Method | Path | Description |
|
||||
| ------ | -------------------------------- | ----------------------------------------- |
|
||||
| GET | `/api/plugins` | List installed plugins |
|
||||
| POST | `/api/plugins/install` | Install a plugin from a local path or URL |
|
||||
| DELETE | `/api/plugins/[name]` | Uninstall a plugin |
|
||||
| POST | `/api/plugins/[name]/activate` | Activate a plugin |
|
||||
| POST | `/api/plugins/[name]/deactivate` | Deactivate a plugin |
|
||||
| GET | `/api/plugins/[name]/config` | Get plugin configuration |
|
||||
| PUT | `/api/plugins/[name]/config` | Update plugin configuration |
|
||||
| Method | Path | Description |
|
||||
| ------ | ---------------------------------- | ------------------------------------- |
|
||||
| GET | `/api/plugins` | List installed plugins |
|
||||
| POST | `/api/plugins/marketplace/install` | Install a plugin from the marketplace |
|
||||
| DELETE | `/api/plugins/[name]` | Uninstall a plugin |
|
||||
| POST | `/api/plugins/[name]/activate` | Activate a plugin |
|
||||
| POST | `/api/plugins/[name]/deactivate` | Deactivate a plugin |
|
||||
| GET | `/api/plugins/[name]/config` | Get plugin configuration |
|
||||
| PUT | `/api/plugins/[name]/config` | Update plugin configuration |
|
||||
|
||||
**Auth:** Requires management session.
|
||||
|
||||
|
||||
@@ -21,15 +21,15 @@ lastUpdated: 2026-08-26
|
||||
| **+ deposit-unlock boost** | **+~24M** | A one-time **$10** OpenRouter top-up raises its free pool from 50 → 1000 req/day. Reported separately so it never inflates the steady number. |
|
||||
| Theoretical ceiling (all rate limits, 24/7) | ~10B | Sum of every provider rate limit extrapolated to non-stop use. **Not a guarantee** — do not headline this. |
|
||||
|
||||
**Honest headline:** _OmniRoute aggregates **~1.51B documented free tokens per month** (up to ~2.13B in your first month with signup credits) across 39 free-tier pools — plus a long tail of permanently-free, no-cap providers — and RTK + Caveman compression (15–95% token savings) stretches that further._
|
||||
**Honest headline:** _OmniRoute aggregates **~1.51B documented free tokens per month** (up to ~2.13B in your first month with signup credits) across 38 free-tier pools — plus a long tail of permanently-free, no-cap providers — and RTK + Caveman compression (15–95% token savings) stretches that further._
|
||||
|
||||
> **Why this dropped from the previous ~1.94B.** The 2026-06-17 refresh is an honesty correction, not a loss: `gemini` is now pool-deduped (was inflated by counting each Flash variant separately, 462M → 60M), `cloudflare-ai` corrected to its real 10k-Neurons/day (122M → 30M), `doubao` reclassified as a one-time signup credit (not recurring), and shut-down tiers removed (`chutes`/`phind`/`kluster` discontinued). Partly offset by `llm7` (correct 5M/day → 150M) and new free providers (Kilo, OpenCode Zen, Z.AI GLM-Flash).
|
||||
>
|
||||
> **Further corrected to ~1.37B in v3.8.42:** `longcat` was reclassified from a 150M/mo recurring grant to a one-time 10M signup credit after its free preview ended. Same honesty rule — no provider was dropped by mistake.
|
||||
>
|
||||
> **Updated on 2026-08-26 after retiring Felo Web:** the source now reports 39 recurring pool keys. Felo Web is excluded while its GPL-derived provenance/licensing remains on HOLD. This is the live, CI-gated number (`check:docs-counts` fails the build if this drifts from `computeFreeModelTotals()`).
|
||||
> **Updated on 2026-08-26 after retiring Felo Web:** the source now reports 38 recurring pool keys. Felo Web is excluded while its GPL-derived provenance/licensing remains on HOLD. This is the live, CI-gated number (`check:docs-counts` fails the build if this drifts from `computeFreeModelTotals()`).
|
||||
|
||||
Biggest **documented** contributors: `mistral` 1.00B, `llm7` 150M, `groq` 117M, `gemini` 60M, `cerebras` 30M, `cloudflare-ai` 30M, `sambanova` 30M. (`longcat` is excluded — its 10M LongCat-2.0 grant is a one-time, KYC-gated signup credit, not a recurring monthly budget.)
|
||||
Biggest **documented** contributors: `mistral` 1.00B, `llm7` 150M, `nara` 150M, `gemini` 60M, `cerebras` 30M, `cloudflare-ai` 30M, `api-airforce` 24M. (`longcat` is excluded — its 10M LongCat-2.0 grant is a one-time, KYC-gated signup credit, not a recurring monthly budget.)
|
||||
|
||||
> ⚠️ The theoretical ceiling (~10B) is inflated by rate-limit-only providers with **no published token cap** (`tencent`, `siliconflow`, `nvidia`, `baidu`, `glm-cn`, `sparkdesk`) whose figures would be `RPM/TPM × 24/7 × 30d` — a theoretical maximum no single account will sustain. They are **excluded** from the defensible number (shown in the "permanently free, no cap" row instead). This is the same inflation that makes competitors' multi-billion claims unreliable.
|
||||
|
||||
@@ -175,9 +175,9 @@ A 50-agent web-research pass (official docs + last-7-days news, adversarially ve
|
||||
| `mistral` | recurring | ~1.00B | — | caution | 5 |
|
||||
| `llm7` | recurring | ~150M | — | caution | 4 |
|
||||
| `longcat` | one-time | — | 10M | caution | 1 |
|
||||
| `gemini` | recurring | ~60M | — | caution | 6 |
|
||||
| `gemini` | recurring | ~60M | — | caution | 4 |
|
||||
| `cerebras` | recurring | ~30M | — | caution | 2 |
|
||||
| `cloudflare-ai` | recurring | ~30M | — | caution | 6 |
|
||||
| `cloudflare-ai` | recurring | ~30M | — | caution | 9 |
|
||||
| `api-airforce` | recurring | ~24M | — | caution | 7 |
|
||||
| `ollama-cloud` | recurring | ~20M | — | ambiguous | 8 |
|
||||
| `groq` | recurring | ~15M | — | caution | 5 |
|
||||
|
||||
@@ -25,6 +25,7 @@ lastUpdated: 2026-06-28
|
||||
| `auto/offline` | offline | Favors providers with highest quota availability |
|
||||
| `auto/smart` | smart | Quality-first + higher exploration rate (10%) for better model discovery |
|
||||
| `auto/lkgp` | lkgp | Explicit LKGP (same as default `auto`) |
|
||||
| `auto/chaos` | chaos | Fault-injection weights for resilience testing (chaos engineering) |
|
||||
|
||||
### Category × Tier Composition (`auto/<category>:<tier>`)
|
||||
|
||||
@@ -211,7 +212,7 @@ The Auto-Combo Engine dynamically selects the best provider/model for each reque
|
||||
|
||||
## Mode Packs
|
||||
|
||||
Four pre-defined weight profiles in `open-sse/services/autoCombo/modePacks.ts`. Each pack overrides the default weights to bias selection toward a specific goal. Below are the **full weight tables per pack** (each row sums to 1.0).
|
||||
Six pre-defined weight profiles in `open-sse/services/autoCombo/modePacks.ts` — `ship-fast`, `cost-saver`, `quality-first`, `offline-friendly`, `reliability-first` and `chaos-mode` (fault-injection). Each pack overrides the default weights to bias selection toward a specific goal; the seed weights below are renormalized to sum 1.0 at runtime together with the session/context factors every pack also sets. The table shows the four original packs — see `modePacks.ts` for `reliability-first` and `chaos-mode`.
|
||||
|
||||
| Factor | ship-fast | cost-saver | quality-first | offline-friendly |
|
||||
| :----------- | :-------- | :--------- | :------------ | :--------------- |
|
||||
@@ -225,7 +226,7 @@ Four pre-defined weight profiles in `open-sse/services/autoCombo/modePacks.ts`.
|
||||
|
||||
Notes:
|
||||
|
||||
- `tierAffinity` and `specificityMatch` are not set in mode packs — `calculateScore()` treats them as `?? 0` when absent.
|
||||
- `tierAffinity` and `specificityMatch` are explicitly set to `0` in every mode pack.
|
||||
- Each pack's emphasis at a glance:
|
||||
- **ship-fast** → latencyInv 0.32 + health 0.28 (low-latency, healthy connections)
|
||||
- **cost-saver** → costInv 0.37 (cheapest tokens win)
|
||||
|
||||
@@ -361,7 +361,7 @@ can be run standalone.
|
||||
|
||||
## DB Schema Summary
|
||||
|
||||
Three tables added by migrations `073–075`:
|
||||
Three tables added by migrations `078`, `079` and `085`:
|
||||
|
||||
- `quota_pools` + `quota_allocations` — pool definitions and per-key allocations.
|
||||
- `quota_consumption` — rolling 2-bucket counters per `(apiKeyId, dimensionKey)`.
|
||||
|
||||
@@ -39,7 +39,7 @@ Turn N+1 (client sends follow-up):
|
||||
→ upstream sees consistent history → no 400
|
||||
```
|
||||
|
||||
Capture happens in `open-sse/handlers/chatCore.ts` (two sites, around lines 4093 and 4380). Replay happens in `open-sse/translator/index.ts` after schema coercion but before dispatch.
|
||||
Capture happens in `open-sse/handlers/chatCore.ts` (two sites, at the two `cacheReasoningFromAssistantMessage` call sites). Replay happens in `open-sse/translator/index.ts` after schema coercion but before dispatch.
|
||||
|
||||
## Storage — Hybrid Memory + SQLite
|
||||
|
||||
@@ -47,7 +47,7 @@ The hot path uses an in-memory `Map` (LRU-by-creation) backed by a SQLite table
|
||||
|
||||
| Layer | Implementation | Purpose |
|
||||
| ------ | ---------------------------------------------- | -------------------------------------- |
|
||||
| Memory | `Map` in `open-sse/services/reasoningCache.ts` | Fast lookups, evicts oldest at 2000 |
|
||||
| Memory | `Map` in `open-sse/services/reasoningCache.ts` | Fast lookups, evicts oldest at 200 |
|
||||
| DB | `reasoning_cache` table (`src/lib/db/`) | Persists across restarts, drives stats |
|
||||
|
||||
Writes go to both. Reads consult memory first, then fall back to DB (DB hits are promoted back into memory). DB failures are non-fatal — the in-memory cache continues to serve the hot path.
|
||||
@@ -55,7 +55,7 @@ Writes go to both. Reads consult memory first, then fall back to DB (DB hits are
|
||||
**Defaults:**
|
||||
|
||||
- TTL: `2h` (`TTL_MS = 2 * 60 * 60 * 1000`)
|
||||
- Max memory entries: `2000` (`MAX_MEMORY_ENTRIES`)
|
||||
- Max memory entries: `200` (`MAX_MEMORY_ENTRIES`)
|
||||
- Eviction: oldest `createdAt` first
|
||||
|
||||
## Database Schema
|
||||
|
||||
@@ -3,11 +3,11 @@
|
||||
<rect x="16" y="16" width="868" height="520" rx="13" fill="#161b22" stroke="#30363d"/>
|
||||
<text x="868" y="528" fill="#484f58" font-size="10.5" text-anchor="end">OmniRoute · /dashboard/free-tiers · preview mockup</text>
|
||||
<text x="32" y="50" fill="#e6edf3" font-size="18" font-weight="700">Monthly free-token budget</text>
|
||||
<text x="868" y="50" fill="#7d8590" font-size="13" text-anchor="end">18 free pools · 425 models · one endpoint</text>
|
||||
<text x="868" y="50" fill="#7d8590" font-size="13" text-anchor="end">20 free pools · 446 models · one endpoint</text>
|
||||
<text x="32" y="84" fill="#7d8590" font-size="11.5">Steady / month</text>
|
||||
<text x="32" y="114" fill="#e6edf3" font-size="27" font-weight="800">~1.46B</text>
|
||||
<text x="32" y="114" fill="#e6edf3" font-size="27" font-weight="800">~1.51B</text>
|
||||
<text x="330" y="84" fill="#7d8590" font-size="11.5">First month (+ signup credits)</text>
|
||||
<text x="330" y="114" fill="#3fb950" font-size="27" font-weight="800">~2.09B</text>
|
||||
<text x="330" y="114" fill="#3fb950" font-size="27" font-weight="800">~2.13B</text>
|
||||
<text x="700" y="84" fill="#7d8590" font-size="11.5">ToS-flagged (you decide)</text>
|
||||
<text x="700" y="114" fill="#d29922" font-size="27" font-weight="800">13 providers</text>
|
||||
<clipPath id="bar"><rect x="32" y="132" width="836" height="16" rx="8"/></clipPath>
|
||||
|
||||
|
Before Width: | Height: | Size: 7.6 KiB After Width: | Height: | Size: 7.6 KiB |
@@ -39,25 +39,41 @@ spawn-capable route: a leaked token over a tunnel still can't reach the spawn.
|
||||
`check-route-guard-membership` gate enumerates every `route.ts` under the
|
||||
spawn-capable prefixes and fails CI if any is not classified local-only.
|
||||
|
||||
| Prefix / pattern | Why it's local-only | Manage-scope bypassable? |
|
||||
| -------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------- |
|
||||
| `/api/mcp/` | MCP server — spawns stdio bridges + SSE handlers | **Yes** (only one) |
|
||||
| `/api/cli-tools/runtime/` | CLI tool runtime — executes arbitrary plugin code | No — spawn-capable |
|
||||
| `/api/modality-bridge/video/` | Strict trusted-loopback Video Bridge runtime probe and authenticated internal extraction broker — fixed FFmpeg/ffprobe invocations with bounded bytes/queue/output | No — spawn-capable |
|
||||
| `/api/services/` | Embedded services (9router/CLIProxy) — `npm install` + spawn | No — spawn-capable |
|
||||
| `/dashboard/providers/services/` | Reverse proxy to embedded-service UIs | No |
|
||||
| `/api/copilot/` | Unauthenticated LLM driver — CLI-only by default | Operator opt-in: manage/admin |
|
||||
| `/api/tools/agent-bridge/` | AgentBridge — spawns MITM server + DNS edits | No — spawn-capable |
|
||||
| `/api/tools/traffic-inspector/` | Traffic Inspector — http-proxy listener + system proxy | No — spawn-capable |
|
||||
| `/api/plugins/`, `/api/plugins` | Plugins — load/execute via `worker_threads` + `child_process` | No — spawn-capable |
|
||||
| `/api/system/version` | Auto-update (POST only; GET/HEAD/OPTIONS exempt) — spawns `git checkout` + `npm install` | No |
|
||||
| `/api/db-backups/exportAll` | Spawns `tar` for the export archive | No |
|
||||
| `/api/local/` | 1-click local launchers (Redis today) — spawns podman/docker | No — spawn-capable |
|
||||
| `/api/headroom/start`, `/stop` | Headroom proxy lifecycle — spawns python CLI / signals PID | No — spawn-capable |
|
||||
| `/api/oauth/cursor/auto-import` | `execFile("which", ["cursor"])` before importing creds | No |
|
||||
| `/api/providers/{id}/login` (regex) | Launches a headful Playwright Chromium for web-cookie login | No |
|
||||
| `/api/providers/{id}/refresh-cursor` (regex) | Manual Cursor session renewal — nudges `cursor-agent` (`--list-models`/`status` via `src/lib/cursor/renewal.ts`); the rest of `/api/providers/`, including the generic `/refresh`, intentionally stays remote-reachable | No — spawn-capable |
|
||||
| `/api/providers/cursor/agent-availability` | Dashboard install-nudge check — spawns `cursor-agent status --format json` via `checkCursorAgentAvailability()`/`getCachedCursorAgentAvailability()` (`src/lib/cursor/renewal.ts`); credential-free response (`{cursorAgentAvailable: boolean}` only) | No — spawn-capable |
|
||||
| Prefix / pattern | Why it's local-only |
|
||||
| -------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- |
|
||||
| `/api/mcp/` | MCP server — spawns stdio bridges + SSE handlers |
|
||||
| `/api/cli-tools/runtime/` | CLI tool runtime — executes arbitrary plugin code |
|
||||
| `/api/cli-tools/{omp,letta,grok-build,forge,jcode,qwen}-settings` | Per-tool settings writers that can touch tool binaries/config on the host |
|
||||
| `/api/cli-tools/antigravity-mitm` | Antigravity MITM proxy control (spawns/points system proxy) |
|
||||
| `/api/modality-bridge/video/` | Strict trusted-loopback Video Bridge runtime probe and internal extraction bridge |
|
||||
| `/api/services/` | Embedded services (9Router / CLIProxy / Bifrost / Mux / Dario) — `npm install` + spawn |
|
||||
| `/dashboard/providers/services/` | Reverse proxy to embedded-service UIs |
|
||||
| `/api/tunnels/cloudflared` | Installs/spawns the cloudflared binary |
|
||||
| `/api/tunnels/tailscale/{install,enable,disable,login,start-daemon}` | Installs/controls tailscaled on the host |
|
||||
| `/api/copilot/` | Unauthenticated LLM driver — CLI-only by default |
|
||||
| `/api/tools/agent-bridge/` | AgentBridge — spawns MITM server + DNS edits |
|
||||
| `/api/tools/traffic-inspector/` | Traffic Inspector — http-proxy listener + system proxy |
|
||||
| `/api/settings/mitm` | Enables MITM interception (system-level proxy state) |
|
||||
| `/api/issue-agent/` | Issue agent — spawns local tooling against the repo |
|
||||
| `/api/plugins/`, `/api/plugins` | Plugins — load/execute via `worker_threads` + `child_process` |
|
||||
| `/api/middleware/` | User middleware — loads/executes operator code in-process |
|
||||
| `/api/system/version` | Auto-update (POST only; GET/HEAD/OPTIONS exempt) — spawns `git checkout` + `npm install` |
|
||||
| `/api/db-backups/exportAll` | Spawns `tar` for the export archive |
|
||||
| `/api/local/` | 1-click local launchers (Redis today) — spawns podman/docker |
|
||||
| `/api/headroom/start`, `/api/headroom/stop` | Headroom proxy lifecycle — spawns python CLI / signals PID |
|
||||
| `/api/jobs`, `/api/jobs/` | Job runner control — executes scheduled host-side work |
|
||||
| `/api/oauth/cursor/auto-import` | `execFile("which", ["cursor"])` before importing creds |
|
||||
| `/api/oauth/kiro/auto-import` | Reads Kiro CLI credential files from the host |
|
||||
| `/api/skills/collect/` | Skill collection — detects/installs local tooling |
|
||||
| `/api/discovery/` | Local network/provider discovery probes |
|
||||
| `/api/vnc-session` (`VNC_ROUTE_PREFIX`) | Spawns a headful browser + VNC session for interactive logins |
|
||||
| `/api/acp/agents` | ACP — discovers and spawns local CLI agent binaries |
|
||||
| `/api/resilience/connections`, `/dashboard/resilience/connections` | Connection maintenance actions that can touch local CLI state |
|
||||
| `/api/providers/cursor/agent-availability` | Dashboard install-nudge check — spawns `cursor-agent status --format json` |
|
||||
| `/api/providers/{id}/login` (regex) | Launches a headful Playwright Chromium for web-cookie login |
|
||||
| `/api/providers/volcengine-plan/connect` (regex) | Manual headful flow + session-based phone/SMS auto-login (spawns Playwright) |
|
||||
| `/api/providers/{id}/refresh-cursor` (regex) | Manual Cursor session renewal — nudges `cursor-agent` |
|
||||
| `/api/providers/{id}/chatgpt-web-codex-doctor` (regex) | Diagnoses the local Codex CLI install (spawns the binary) |
|
||||
|
||||
**Response on violation:** `403 LOCAL_ONLY`
|
||||
|
||||
@@ -142,10 +158,14 @@ These routes are destructive or irreversible. Allowing them in a "no-password"
|
||||
install would mean anyone on the same LAN could wipe the database or kill the
|
||||
server process.
|
||||
|
||||
| Path | Reason |
|
||||
| ------------------------ | --------------------------------- |
|
||||
| `/api/shutdown` | Terminates the server process |
|
||||
| `/api/settings/database` | Database export, import, and wipe |
|
||||
| Path | Reason |
|
||||
| ----------------------------------------- | ---------------------------------------------- |
|
||||
| `/api/shutdown` | Terminates the server process |
|
||||
| `/api/settings/database` | Database export, import, and wipe |
|
||||
| `/api/db-backups` | Full database backup archive access |
|
||||
| `/api/settings/export-json` | Exports the full settings blob (incl. secrets) |
|
||||
| `/api/settings/import-json` | Replaces the full settings blob |
|
||||
| `/api/providers/health-autopilot/actions` | Executes autopilot remediation actions |
|
||||
|
||||
**Response on violation:** `401 Authentication required`
|
||||
|
||||
|
||||
@@ -74,7 +74,7 @@ Applied to: `system` blocks, all `messages[].content`, and `tools[].description`
|
||||
|
||||
For third-party Anthropic relays that only accept "real Claude Code" traffic:
|
||||
|
||||
- `CLAUDE_CODE_COMPATIBLE_USER_AGENT = "claude-cli/2.1.219 (external, sdk-cli)"`
|
||||
- `CLAUDE_CODE_COMPATIBLE_USER_AGENT = "claude-cli/2.1.220 (external, sdk-cli)"`
|
||||
- `CLAUDE_CODE_COMPATIBLE_STAINLESS_PACKAGE_VERSION = "0.94.0"`
|
||||
- `CLAUDE_CODE_COMPATIBLE_STAINLESS_RUNTIME_VERSION = "v26.3.0"`
|
||||
- `anthropic-beta = "claude-code-20250219,interleaved-thinking-2025-05-14,effort-2025-11-24"` by default
|
||||
@@ -198,8 +198,8 @@ All MITM endpoints require management auth (`requireCliToolsAuth`). The sudo pas
|
||||
|
||||
| Variable | Default |
|
||||
| ------------------------ | --------------------------------------------------------------- |
|
||||
| `CLAUDE_USER_AGENT` | `claude-cli/2.1.219 (external, cli)` |
|
||||
| `CODEX_USER_AGENT` | `codex-cli/0.142.0 (Windows 10.0.26200; x64)` |
|
||||
| `CLAUDE_USER_AGENT` | `claude-cli/2.1.220 (external, cli)` |
|
||||
| `CODEX_USER_AGENT` | `codex-cli/0.149.0 (Windows 10.0.26200; x64)` |
|
||||
| `GITHUB_USER_AGENT` | `GitHubCopilotChat/0.54.0` |
|
||||
| `ANTIGRAVITY_USER_AGENT` | `antigravity/2.0.1 linux/arm64 google-api-nodejs-client/10.3.0` |
|
||||
| `KIRO_USER_AGENT` | `AWS-SDK-JS/3.0.0 kiro-ide/1.0.0` |
|
||||
|
||||
71
llm.txt
71
llm.txt
@@ -8,11 +8,11 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.8.50
|
||||
**Current version:** 3.8.51
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Runtime:** Node.js `>=22.0.0 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Runtime:** Node.js `>=22.22.2 <23 || >=24.0.0 <27`, ES Modules (`"type": "module"`)
|
||||
- **Framework:** Next.js 16 (App Router) with TypeScript 6
|
||||
- **Database:** SQLite via better-sqlite3 (local, zero-config, 167 migrations)
|
||||
- **State management:** Zustand (client), SQLite (server persistence)
|
||||
@@ -57,7 +57,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── skills/ # Skills system dashboard
|
||||
│ │ │ ├── translator/ # Format translator + debug tools
|
||||
│ │ │ └── usage/ # Usage history
|
||||
│ │ ├── api/ # REST API endpoints (51 route directories)
|
||||
│ │ ├── api/ # REST API endpoints (100+ route directories)
|
||||
│ │ │ ├── v1/ # OpenAI-compatible API (chat, completions, models, embeddings,
|
||||
│ │ │ │ # images, audio, videos, music, moderations, rerank, search,
|
||||
│ │ │ │ # responses, messages, registered-keys, quotas, accounts)
|
||||
@@ -102,7 +102,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ └── streaming.ts # SSE streaming for A2A
|
||||
│ │ ├── acp/ # Agent Communication Protocol registry and manager
|
||||
│ │ ├── compliance/ # Compliance policy engine
|
||||
│ │ ├── db/ # SQLite database layer (117 modules + migrations)
|
||||
│ │ ├── db/ # SQLite database layer (122 modules + migrations)
|
||||
│ │ │ ├── core.ts # Database initialization, connection, schema
|
||||
│ │ │ ├── providers.ts # Provider connection CRUD
|
||||
│ │ │ ├── models.ts # Model catalog management
|
||||
@@ -124,7 +124,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ │ ├── secrets.ts # Secrets management
|
||||
│ │ │ ├── stateReset.ts # State reset utilities
|
||||
│ │ │ ├── migrationRunner.ts # Schema migration runner
|
||||
│ │ │ └── migrations/ # 149 versioned SQL migration files
|
||||
│ │ │ └── migrations/ # 167 versioned SQL migration files
|
||||
│ │ ├── evals/ # Eval runner and scheduler
|
||||
│ │ ├── memory/ # Persistent conversational memory
|
||||
│ │ │ ├── extraction.ts # Memory extraction from conversations
|
||||
@@ -165,7 +165,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ └── manager.ts # MITM proxy manager
|
||||
│ ├── shared/ # Shared utilities, components, and constants
|
||||
│ │ ├── components/ # Reusable UI components (Card, Badge, Button, Modal, Sidebar, ProviderIcon, etc.)
|
||||
│ │ ├── constants/ # Provider definitions (349), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── constants/ # Provider definitions (352), model lists, pricing, routing strategies, MCP scopes
|
||||
│ │ ├── contracts/ # Shared API contracts
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ ├── middleware/ # Shared middleware utilities
|
||||
@@ -182,7 +182,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
├── open-sse/ # Standalone SSE server (npm workspace)
|
||||
│ ├── config/ # Model registries (providerRegistry, embedding, image, audio, video,
|
||||
│ │ # music, rerank, moderation, search, CLI fingerprints, Ollama models)
|
||||
│ ├── executors/ # Provider-specific request executors (101 executor modules)
|
||||
│ ├── executors/ # Provider-specific request executors (104 executor modules)
|
||||
│ │ ├── base.ts # Base executor with shared logic
|
||||
│ │ ├── default.ts # Default OpenAI-compatible executor
|
||||
│ │ ├── cursor.ts # Cursor IDE (protobuf + checksum)
|
||||
@@ -195,7 +195,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── cloudflare-ai.ts # Cloudflare Workers AI
|
||||
│ │ ├── opencode.ts # OpenCode Zen/Go
|
||||
│ │ └── pollinations.ts # Pollinations AI
|
||||
│ ├── handlers/ # Request handlers per API type (11 handlers)
|
||||
│ ├── handlers/ # Request handlers per API type (25 handlers)
|
||||
│ │ ├── chatCore.ts # Main chat completions handler
|
||||
│ │ ├── responsesHandler.ts # OpenAI Responses API handler
|
||||
│ │ ├── embeddings.ts # Embedding generation
|
||||
@@ -215,7 +215,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── audit.ts # Tool call audit logging
|
||||
│ │ ├── runtimeHeartbeat.ts # MCP runtime heartbeat
|
||||
│ │ └── httpTransport.ts # HTTP transport handler
|
||||
│ ├── services/ # 140+ service modules
|
||||
│ ├── services/ # 220+ service modules
|
||||
│ │ ├── combo.ts # Core routing engine
|
||||
│ │ ├── usage.ts # Usage tracking
|
||||
│ │ ├── tokenRefresh.ts # OAuth token refresh
|
||||
@@ -223,7 +223,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── accountFallback.ts # Multi-account fallback
|
||||
│ │ ├── sessionManager.ts # Session management
|
||||
│ │ ├── wildcardRouter.ts # Wildcard model routing
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (14-factor scoring, bandit exploration)
|
||||
│ │ ├── autoCombo/ # Auto-combo engine (15-factor scoring, bandit exploration)
|
||||
│ │ ├── intentClassifier.ts # Request intent classification
|
||||
│ │ ├── taskAwareRouter.ts # Task-aware routing
|
||||
│ │ ├── thinkingBudget.ts # Thinking budget management
|
||||
@@ -246,13 +246,13 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ │ ├── response/ # Response translators per provider
|
||||
│ │ ├── helpers/ # Translation helpers
|
||||
│ │ └── image/ # Image format translation
|
||||
│ └── utils/ # 22 utility modules (stream, TLS, proxy, logging, etc.)
|
||||
│ └── utils/ # 100+ utility modules (stream, TLS, proxy, logging, etc.)
|
||||
├── electron/ # Electron desktop app (cross-platform)
|
||||
│ ├── main.js # Electron main process
|
||||
│ ├── preload.js # Preload script (IPC bridge)
|
||||
│ └── assets/ # App icons and assets
|
||||
├── tests/ # Test suites
|
||||
│ ├── unit/ # 2,700+ unit test files
|
||||
│ ├── unit/ # 5,000+ unit test files
|
||||
│ ├── integration/ # Integration tests
|
||||
│ ├── e2e/ # Playwright E2E tests
|
||||
│ ├── security/ # Security tests
|
||||
@@ -263,7 +263,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
│ ├── architecture/ # ARCHITECTURE.md, CODEBASE_DOCUMENTATION.md, REPOSITORY_MAP.md, AUTHZ_GUIDE.md, RESILIENCE_GUIDE.md, QUALITY_GATES.md
|
||||
│ ├── reference/ # API_REFERENCE.md, PROVIDER_REFERENCE.md, CLI-TOOLS.md
|
||||
│ ├── frameworks/ # MCP-SERVER.md (110 tools), A2A-SERVER.md, SKILLS.md, MEMORY.md, CLOUD_AGENT.md, EVALS.md, WEBHOOKS.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (14-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── routing/ # AUTO-COMBO.md (15-factor scoring), REASONING_REPLAY.md
|
||||
│ ├── security/ # GUARDRAILS.md, COMPLIANCE.md, STEALTH_GUIDE.md, PUBLIC_CREDS.md, ERROR_SANITIZATION.md
|
||||
│ ├── guides/ # USER_GUIDE.md, TROUBLESHOOTING.md, ELECTRON_GUIDE.md, I18N.md
|
||||
│ ├── ops/ # RELEASE_CHECKLIST.md, TUNNELS_GUIDE.md, VM deployment
|
||||
@@ -279,19 +279,19 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
### Core Proxy
|
||||
- **352 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **19 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, cache-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
- **Context Relay strategy**: Session handoff summaries on account rotation for continuity
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **14-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Auto-combo engine**: Self-healing routing optimization with **15-factor scoring** (see `docs/routing/AUTO-COMBO.md`), bandit exploration, progressive cooldown
|
||||
- **Semantic caching** with cache hit/miss headers
|
||||
- **Idempotency** with configurable dedup window
|
||||
- **3-layer resilience**: Provider Circuit Breaker / Connection Cooldown / Model Lockout
|
||||
- **Provider Icons**: 130+ provider logos via `@lobehub/icons` (SVG) with PNG fallback
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists for 16 providers
|
||||
- **Model Auto-Sync**: 24h scheduler refreshes model lists across connected providers
|
||||
- **Registered Keys API**: Auto-provision API keys via `POST /api/v1/registered-keys` with quota enforcement
|
||||
- **Memory System**: Persistent conversational memory with extraction, injection, retrieval, and summarization
|
||||
- **Skills System**: Extensible skill framework with registry, executor, sandbox, built-in and custom skills
|
||||
- **Cloud Agents**: Codex Cloud, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Cloud Agents**: Codex Cloud, Cursor, Devin, Jules — autonomous coding agents with task lifecycle management
|
||||
- **Guardrails Framework**: Hot-reloadable registry with vision-bridge, pii-masker, prompt-injection (priority-ordered)
|
||||
- **MITM Proxy**: Certificate management, DNS handling, and target routing
|
||||
- **Cloudflare Tunnels**: Managed tunnel creation for remote access
|
||||
@@ -309,7 +309,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Provider constants validated at module load** via Zod (`src/shared/validation/providerSchema.ts`)
|
||||
- **PII sanitizer** — Sensitive data scrubbing in logs
|
||||
|
||||
### Dashboard Pages (23 sections)
|
||||
### Dashboard Pages (50+ sections)
|
||||
- **Providers** — OAuth, API key, and free provider management with ProviderIcon SVG icons
|
||||
- **Combos** — Multi-model combo builder with 4 templates (Free Stack, High Availability, Cost Saver, Balanced) + 19 strategies
|
||||
- **Auto-Combo** — Auto-combo engine dashboard with scoring metrics
|
||||
@@ -341,35 +341,34 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
- **Gemini** — `/v1beta/models`, `/v1beta/models/{...path}`
|
||||
- **Ollama** — `/v1/api/chat`, `/api/tags`
|
||||
- **Search** — `/v1/search` (Perplexity, Serper, Brave, Exa, Tavily)
|
||||
- **MCP** — 105-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **MCP** — 110-tool MCP server with scope-based auth (3 transports: stdio, SSE, streamable HTTP)
|
||||
- **A2A** — Agent-to-Agent v0.3 protocol (JSON-RPC 2.0, 6 skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities)
|
||||
- **ACP** — Agent Communication Protocol registry and manager
|
||||
|
||||
### MCP Server (109 Tools)
|
||||
### MCP Server (110 Tools)
|
||||
|
||||
110 tools across modules: **44 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
110 tools across modules: **45 canonical** (health, combos, quotas, routing, cost, models, cache,
|
||||
diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool**, **notion**,
|
||||
**obsidian**, **localCorpus**, **gamification**, and **plugin** modules. Full per-tool inventory:
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
**MCP Auth Scopes (32):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
**MCP Auth Scopes (33):** e.g. `read:health`, `read:combos`, `write:combos`, `read:quota`,
|
||||
`read:usage`, `read:models`, `execute:completions`, `execute:search`, `write:budget`,
|
||||
`write:resilience`, plus memory/skills/pool/plugin scopes — full list in
|
||||
`docs/frameworks/MCP-SERVER.md`.
|
||||
|
||||
### Provider Categories
|
||||
|
||||
**Free Providers (3):** Qoder AI, Kiro AI, Windsurf
|
||||
|
||||
**OAuth Providers (13):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
**OAuth Providers (24):** Claude Code, Antigravity, OpenAI Codex, GitHub Copilot, Cursor IDE, Kimi Coding, Kilo Code, Cline, Kiro, Qoder, Gemini, Windsurf, GitLab Duo
|
||||
|
||||
**API Key Providers (48+):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
**API Key Providers (236):** OpenAI, Anthropic, Gemini (Google AI Studio), DeepSeek, Groq, xAI (Grok), Mistral, Perplexity, Together AI, Fireworks AI, Cerebras, Cohere, NVIDIA NIM, Nebius AI, SiliconFlow, Hyperbolic, HuggingFace, OpenRouter, Vertex AI, Cloudflare Workers AI, Scaleway AI, AI/ML API, Pollinations AI, LongCat AI, Alibaba, Alibaba (China), Kimi, Kimi Coding (API Key), Minimax, Minimax (China), Blackbox AI, Synthetic, Kilo Gateway, Z.AI, GLM Coding, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld, NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper Search, Brave Search, Exa Search, Tavily Search, OpenCode Zen, OpenCode Go, Alibaba Coding Plan
|
||||
|
||||
**Custom Providers:** OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) with custom base URLs
|
||||
|
||||
### Internationalization
|
||||
- 43 languages for UI (all dashboard pages)
|
||||
- 40 translated documentation sets in docs/i18n/
|
||||
- 42 translated documentation sets in docs/i18n/
|
||||
- Language switcher in documentation
|
||||
|
||||
## Key Architectural Decisions
|
||||
@@ -384,13 +383,13 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
5. **SSE proxy pipeline:** The proxy pipeline is middleware-based: request → auth resolution → rate limiting → circuit breaker → format translation → upstream call → response translation → SSE streaming back to client.
|
||||
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 99 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
6. **SQLite for persistence:** All state (providers, combos, logs, settings, API keys, memory, skills) stored in a single SQLite database via 122 domain-specific modules. All DB operations go through `src/lib/db/` modules, never raw SQL in routes.
|
||||
|
||||
7. **OAuth with PKCE:** OAuth flows use PKCE for security. Token refresh handled by background job (`tokenHealthCheck.ts`).
|
||||
|
||||
8. **ProviderIcon component:** Unified icon system using `@lobehub/icons` (130+ SVG) with PNG fallback and generic icon fallback chain. Used on providers, dashboard, and agents pages.
|
||||
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 117 `src/lib/db/` modules with 149 SQL migrations.
|
||||
9. **DB architecture:** `localDb.ts` is a re-export layer only — real logic lives in 122 `src/lib/db/` modules with 167 SQL migrations.
|
||||
|
||||
10. **Upstream headers:** Custom headers merged in executors after default auth; same header name replaces executor value. Forbidden header names in `src/shared/constants/upstreamHeaders.ts`.
|
||||
|
||||
@@ -434,7 +433,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
4. **Environment variables:** All configuration is in `.env` (from `.env.example`). Key vars: `PORT`, `NEXT_PUBLIC_BASE_URL`, `API_KEY`, `ADMIN_PASSWORD`.
|
||||
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (120 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
5. **Database layer:** Operations go through `src/lib/db/` modules (122 domain-specific files, 167 migrations). `localDb.ts` is re-exports only — add new functions to the proper `db/*.ts` module.
|
||||
|
||||
6. **Tests** use Node.js built-in test runner + Vitest. Run `npm test`. Vitest for MCP/autoCombo (`npm run test:vitest`). Playwright for E2E (`npm run test:e2e`). Coverage gate: ratchet vs `quality-baseline.json`, absolute floor 60% statements/lines/functions/branches.
|
||||
|
||||
@@ -442,9 +441,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
8. **ACP agents** are in `src/lib/acp/registry.ts` with detection cache. Custom agents stored via settings DB.
|
||||
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **14-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 4 mode packs, bandit exploration, progressive cooldown.
|
||||
9. **Auto-combo engine** in `open-sse/services/autoCombo/` — **15-factor scoring** (weights and factors in `docs/routing/AUTO-COMBO.md`), 6 mode packs, bandit exploration, progressive cooldown.
|
||||
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (3 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
10. **Docker:** Dockerfile has two targets: `runner-base` and `runner-cli`. `docker-compose.yml` for dev (8 profiles), `docker-compose.prod.yml` for production (port 20130).
|
||||
|
||||
11. **Electron desktop app** in `electron/` with main.js and preload.js. Build with `npm run electron:build` (supports Windows, macOS, Linux).
|
||||
|
||||
@@ -462,9 +461,9 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
18. **Node.js 24+ compatibility**: The login page (`/api/settings/require-login`) detects the Node.js version and sends `nodeVersion`/`nodeCompatible` fields. The login UI renders a warning banner when `nodeCompatible` is false.
|
||||
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — three external autonomous coding agents (Codex Cloud, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
19. **Cloud Agents** in `src/lib/cloudAgent/` — four external autonomous coding agents (Codex Cloud, Cursor, Devin, Jules) with task lifecycle endpoints under `/api/v1/agents/tasks/`. Require management auth, not client auth.
|
||||
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `pii-masker` (10) → `prompt-injection` (20). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
20. **Guardrails framework** in `src/lib/guardrails/` — hot-reloadable registry. Built-ins (priority-ordered): `vision-bridge` (5) → `audio-bridge` (6) → `video-bridge` (7) → `pii-masker` (10) → `prompt-injection` (20) → `credential-masker` (95). Fail-open model: exceptions never block traffic. Per-request opt-out via `x-omniroute-disabled-guardrails` header.
|
||||
|
||||
21. **Authz pipeline** (`src/server/authz/`): every request is classified as `PUBLIC`, `CLIENT_API`, or `MANAGEMENT`, then run through policy + enforce stages. See `docs/architecture/AUTHZ_GUIDE.md`.
|
||||
|
||||
@@ -475,11 +474,11 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **351-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **352-provider catalog** with 150+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **15-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
- **Cloud Agents** (Codex Cloud, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Cloud Agents** (Codex Cloud, Cursor, Devin, Jules), **Guardrails**, **Evals**, **Webhooks**, **Compliance** frameworks
|
||||
- **Embedded services** manager (install/start/stop bundled services from the dashboard)
|
||||
- **Prompt compression** (RTK + Caveman codecs) saving up to ~95% tokens on eligible traffic
|
||||
- Full changelog: `CHANGELOG.md`
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# OmniRoute MCP Server
|
||||
|
||||
> **Model Context Protocol server** that exposes OmniRoute's gateway intelligence as **107 tools** for AI agents.
|
||||
> **Model Context Protocol server** that exposes OmniRoute's gateway intelligence as **110 tools** for AI agents.
|
||||
>
|
||||
> **Source of truth for the full tool catalog and REST surface:** [`docs/frameworks/MCP-SERVER.md`](../../docs/frameworks/MCP-SERVER.md). This README focuses on architecture, configuration, and integration examples; the catalog below is a summary subset.
|
||||
|
||||
|
||||
@@ -1,139 +0,0 @@
|
||||
<svg viewBox="0 0 1200 540" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="The OmniRoute promise: one endpoint, 349 providers — never stop building, OmniRoute picks the cheapest one that works. Six pillars. Never hit limits: auto-fallback across 349 providers in milliseconds, quota out means the next provider takes over with zero downtime. Save up to 95 percent of tokens: RTK plus Caveman stacked compression cuts 15 to 95 percent of eligible tokens, about 89 percent average on tool-heavy sessions. Zero dollars to start: 90+ providers with a free tier, 56 free forever — Qoder, Pollinations, Cloudflare, SiliconFlow — no card needed. Every tool works: 33 coding agents including Claude Code, Codex, Cursor, Cline, Copilot and Antigravity through one config. One endpoint: OpenAI, Claude, Gemini and Responses API translation — point any tool at /v1 and it just works. Production-grade: circuit breakers, TLS stealth, MCP with 110 tools, A2A, memory, guardrails, evals — 25,000+ tests.">
|
||||
<desc>Animated promise card: six pillar tiles fade in in reading order, then a soft colored border highlight sweeps from tile to tile in a continuous cycle.</desc>
|
||||
<defs>
|
||||
<pattern id="gridPaperP" width="32" height="32" patternUnits="userSpaceOnUse">
|
||||
<path d="M 32 0 L 0 0 0 32" fill="none" stroke="#ffffff" stroke-opacity="0.06" stroke-width="1"/>
|
||||
</pattern>
|
||||
<radialGradient id="bgGlowP" cx="70%" cy="14%" r="55%">
|
||||
<stop offset="0%" stop-color="#e54d5e" stop-opacity="0.12"/>
|
||||
<stop offset="55%" stop-color="#8b5cf6" stop-opacity="0.06"/>
|
||||
<stop offset="100%" stop-color="#6366f1" stop-opacity="0"/>
|
||||
</radialGradient>
|
||||
</defs>
|
||||
|
||||
<rect width="1200" height="540" fill="#0b0e14"/>
|
||||
<rect width="1200" height="540" fill="url(#gridPaperP)"/>
|
||||
<rect width="1200" height="540" fill="url(#bgGlowP)"/>
|
||||
|
||||
<!-- header -->
|
||||
<g>
|
||||
<text x="40" y="58" font-family="Consolas, 'Courier New', monospace" font-size="13" letter-spacing="2.5" fill="#a78bfa">THE PROMISE</text>
|
||||
<line x1="150" y1="53" x2="1160" y2="53" stroke="#232b38" stroke-width="1.5"/>
|
||||
</g>
|
||||
<g>
|
||||
<text x="40" y="100" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="23" font-weight="600" fill="#c9d1d9">One endpoint. <tspan fill="#a78bfa" font-weight="800">349 providers.</tspan> Never stop building — OmniRoute picks <tspan fill="#7ee787" font-weight="700">the cheapest one that works</tspan>.</text>
|
||||
</g>
|
||||
|
||||
<g font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif">
|
||||
|
||||
<!-- cell 1: never hit limits (blue) -->
|
||||
<g>
|
||||
<rect x="40" y="130" width="360" height="164" rx="14" fill="#161b22" stroke="#ffffff" stroke-opacity="0.08" stroke-width="1"/>
|
||||
<rect x="40" y="130" width="360" height="164" rx="14" fill="none" stroke="#74b9ff" stroke-width="1.5" opacity="0">
|
||||
<animate attributeName="opacity" values="0;0.9;0;0" keyTimes="0;0.055;0.12;1" dur="12s" begin="3s" repeatCount="indefinite"/>
|
||||
</rect>
|
||||
<rect x="40" y="146" width="4" height="132" rx="2" fill="#0984e3"/>
|
||||
<g transform="translate(66,152)" stroke="#74b9ff" stroke-width="2.4" fill="none" stroke-linecap="round">
|
||||
<circle cx="11" cy="11" r="10.5"/>
|
||||
<line x1="3.9" y1="3.9" x2="18.1" y2="18.1"/>
|
||||
</g>
|
||||
<text x="102" y="170" font-size="18" font-weight="800" fill="#74b9ff">Never hit limits</text>
|
||||
<text x="66" y="204" font-size="13.5" fill="#a1a1aa">Auto-fallback across 349 providers in</text>
|
||||
<text x="66" y="226" font-size="13.5" fill="#a1a1aa">milliseconds. Quota out? The next provider</text>
|
||||
<text x="66" y="248" font-size="13.5" fill="#a1a1aa">takes over — zero downtime.</text>
|
||||
</g>
|
||||
|
||||
<!-- cell 2: save tokens (orange) -->
|
||||
<g>
|
||||
<rect x="420" y="130" width="360" height="164" rx="14" fill="#161b22" stroke="#ffffff" stroke-opacity="0.08" stroke-width="1"/>
|
||||
<rect x="420" y="130" width="360" height="164" rx="14" fill="none" stroke="#e17055" stroke-width="1.5" opacity="0">
|
||||
<animate attributeName="opacity" values="0;0;0.9;0;0" keyTimes="0;0.167;0.222;0.287;1" dur="12s" begin="3s" repeatCount="indefinite"/>
|
||||
</rect>
|
||||
<rect x="420" y="146" width="4" height="132" rx="2" fill="#e17055"/>
|
||||
<g transform="translate(446,152)" stroke="#e17055" stroke-width="2.4" fill="none" stroke-linecap="round">
|
||||
<circle cx="5.5" cy="5.5" r="4"/>
|
||||
<circle cx="16.5" cy="16.5" r="4"/>
|
||||
<line x1="19" y1="3" x2="3" y2="19"/>
|
||||
</g>
|
||||
<text x="482" y="170" font-size="18" font-weight="800" fill="#e17055">Save up to 95% tokens</text>
|
||||
<text x="446" y="204" font-size="13.5" fill="#a1a1aa">RTK + Caveman stacked compression cuts</text>
|
||||
<text x="446" y="226" font-size="13.5" fill="#a1a1aa">15–95% of eligible tokens — ~89% average</text>
|
||||
<text x="446" y="248" font-size="13.5" fill="#a1a1aa">on tool-heavy sessions.</text>
|
||||
</g>
|
||||
|
||||
<!-- cell 3: $0 to start (gold) -->
|
||||
<g>
|
||||
<rect x="800" y="130" width="360" height="164" rx="14" fill="#161b22" stroke="#ffffff" stroke-opacity="0.08" stroke-width="1"/>
|
||||
<rect x="800" y="130" width="360" height="164" rx="14" fill="none" stroke="#fdcb6e" stroke-width="1.5" opacity="0">
|
||||
<animate attributeName="opacity" values="0;0;0.9;0;0" keyTimes="0;0.334;0.389;0.454;1" dur="12s" begin="3s" repeatCount="indefinite"/>
|
||||
</rect>
|
||||
<rect x="800" y="146" width="4" height="132" rx="2" fill="#fdcb6e"/>
|
||||
<g transform="translate(826,152)" stroke="#fdcb6e" stroke-width="2.2" fill="none" stroke-linecap="round" stroke-linejoin="round">
|
||||
<path d="M 2,9.5 L 2,3 A 1,1 0 0 1 3,2 L 9.5,2 L 20,12.5 A 1.6,1.6 0 0 1 20,14.8 L 14.8,20 A 1.6,1.6 0 0 1 12.5,20 Z"/>
|
||||
<circle cx="6.6" cy="6.6" r="1.4" fill="#fdcb6e" stroke="none"/>
|
||||
</g>
|
||||
<text x="862" y="170" font-size="18" font-weight="800" fill="#fdcb6e">$0 to start</text>
|
||||
<text x="826" y="204" font-size="13.5" fill="#a1a1aa">90+ providers with a free tier, 56 free</text>
|
||||
<text x="826" y="226" font-size="13.5" fill="#a1a1aa">forever — Qoder, Pollinations, Cloudflare,</text>
|
||||
<text x="826" y="248" font-size="13.5" fill="#a1a1aa">SiliconFlow… No card needed.</text>
|
||||
</g>
|
||||
|
||||
<!-- cell 4: every tool works (violet) -->
|
||||
<g>
|
||||
<rect x="40" y="314" width="360" height="164" rx="14" fill="#161b22" stroke="#ffffff" stroke-opacity="0.08" stroke-width="1"/>
|
||||
<rect x="40" y="314" width="360" height="164" rx="14" fill="none" stroke="#a78bfa" stroke-width="1.5" opacity="0">
|
||||
<animate attributeName="opacity" values="0;0;0.9;0;0" keyTimes="0;0.5;0.555;0.62;1" dur="12s" begin="3s" repeatCount="indefinite"/>
|
||||
</rect>
|
||||
<rect x="40" y="330" width="4" height="132" rx="2" fill="#8b5cf6"/>
|
||||
<g transform="translate(66,336)" stroke="#a78bfa" stroke-width="2.2" fill="none" stroke-linecap="round" stroke-linejoin="round">
|
||||
<path d="M 6,2 L 6,7 M 14,2 L 14,7"/>
|
||||
<path d="M 3.5,7 L 16.5,7 L 16.5,11.5 A 6.5,6.5 0 0 1 10,18 A 6.5,6.5 0 0 1 3.5,11.5 Z"/>
|
||||
<path d="M 10,18 L 10,22"/>
|
||||
</g>
|
||||
<text x="102" y="354" font-size="18" font-weight="800" fill="#a78bfa">Every tool works</text>
|
||||
<text x="66" y="388" font-size="13.5" fill="#a1a1aa">33 coding agents — Claude Code, Codex,</text>
|
||||
<text x="66" y="410" font-size="13.5" fill="#a1a1aa">Cursor, Cline, Copilot, Antigravity —</text>
|
||||
<text x="66" y="432" font-size="13.5" fill="#a1a1aa">through one config.</text>
|
||||
</g>
|
||||
|
||||
<!-- cell 5: one endpoint (cyan) -->
|
||||
<g>
|
||||
<rect x="420" y="314" width="360" height="164" rx="14" fill="#161b22" stroke="#ffffff" stroke-opacity="0.08" stroke-width="1"/>
|
||||
<rect x="420" y="314" width="360" height="164" rx="14" fill="none" stroke="#00cec9" stroke-width="1.5" opacity="0">
|
||||
<animate attributeName="opacity" values="0;0;0.9;0;0" keyTimes="0;0.667;0.722;0.787;1" dur="12s" begin="3s" repeatCount="indefinite"/>
|
||||
</rect>
|
||||
<rect x="420" y="330" width="4" height="132" rx="2" fill="#00cec9"/>
|
||||
<g transform="translate(446,336)" stroke="#00cec9" stroke-width="2.2" fill="none" stroke-linecap="round">
|
||||
<path d="M 2,3 L 13,8.5 M 2,11 L 13,11 M 2,19 L 13,13.5"/>
|
||||
<circle cx="18.5" cy="11" r="3.2" fill="#00cec9" stroke="none"/>
|
||||
</g>
|
||||
<text x="482" y="354" font-size="18" font-weight="800" fill="#00cec9">One endpoint</text>
|
||||
<text x="446" y="388" font-size="13.5" fill="#a1a1aa">OpenAI ↔ Claude ↔ Gemini ↔ Responses API</text>
|
||||
<text x="446" y="410" font-size="13.5" fill="#a1a1aa">translation. Point any tool at /v1 —</text>
|
||||
<text x="446" y="432" font-size="13.5" fill="#a1a1aa">it just works.</text>
|
||||
</g>
|
||||
|
||||
<!-- cell 6: production-grade (green) -->
|
||||
<g>
|
||||
<rect x="800" y="314" width="360" height="164" rx="14" fill="#161b22" stroke="#ffffff" stroke-opacity="0.08" stroke-width="1"/>
|
||||
<rect x="800" y="314" width="360" height="164" rx="14" fill="none" stroke="#7ee787" stroke-width="1.5" opacity="0">
|
||||
<animate attributeName="opacity" values="0;0;0.9;0;0" keyTimes="0;0.834;0.889;0.954;1" dur="12s" begin="3s" repeatCount="indefinite"/>
|
||||
</rect>
|
||||
<rect x="800" y="330" width="4" height="132" rx="2" fill="#22c55e"/>
|
||||
<g transform="translate(826,336)" stroke="#7ee787" stroke-width="2.2" fill="none" stroke-linecap="round" stroke-linejoin="round">
|
||||
<path d="M 11,1.5 L 19.5,5 L 19.5,11 C 19.5,16.5 16,19.8 11,21.5 C 6,19.8 2.5,16.5 2.5,11 L 2.5,5 Z"/>
|
||||
<path d="M 7,11 L 10,14 L 15.4,8.2"/>
|
||||
</g>
|
||||
<text x="862" y="354" font-size="18" font-weight="800" fill="#7ee787">Production-grade</text>
|
||||
<text x="826" y="388" font-size="13.5" fill="#a1a1aa">Circuit breakers, TLS stealth, MCP (110</text>
|
||||
<text x="826" y="410" font-size="13.5" fill="#a1a1aa">tools), A2A, memory, guardrails, evals —</text>
|
||||
<text x="826" y="432" font-size="13.5" fill="#a1a1aa">25,000+ tests.</text>
|
||||
</g>
|
||||
</g>
|
||||
|
||||
<!-- footer -->
|
||||
<g>
|
||||
<text x="40" y="516" font-family="Consolas, 'Courier New', monospace" font-size="12" fill="#71717a">$ npm i -g omniroute  ·  point your tool at http://localhost:20128/v1  ·  $0</text>
|
||||
<text x="1160" y="516" text-anchor="end" font-family="Consolas, 'Courier New', monospace" font-size="12" letter-spacing="1.5" fill="#71717a">MIT · OPEN SOURCE</text>
|
||||
</g>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 10 KiB |
@@ -13,7 +13,7 @@
|
||||
|
||||
<!-- Title -->
|
||||
<text x="400" y="32" text-anchor="middle" font-size="18" font-weight="700" fill="#f9fafb">OmniRoute 3-tier fallback</text>
|
||||
<text x="400" y="52" text-anchor="middle" font-size="12" fill="#6b7280">Never stop building — automatic zero-config failover across 207+ providers</text>
|
||||
<text x="400" y="52" text-anchor="middle" font-size="12" fill="#6b7280">Never stop building — automatic zero-config failover across 352 providers</text>
|
||||
|
||||
<!-- Client box -->
|
||||
<rect x="275" y="70" width="250" height="52" rx="8" fill="#1f2937" stroke="#374151" stroke-width="1.5"/>
|
||||
@@ -27,8 +27,8 @@
|
||||
<!-- Router box -->
|
||||
<rect x="200" y="152" width="400" height="72" rx="8" fill="#1e3a5f" stroke="#3b82f6" stroke-width="1.5"/>
|
||||
<text x="400" y="177" text-anchor="middle" font-size="14" font-weight="700" fill="#93c5fd">OmniRoute Smart Router</text>
|
||||
<text x="400" y="196" text-anchor="middle" font-size="10" fill="#60a5fa">RTK compression · Caveman terse-mode · Auto-Combo (14 strategies)</text>
|
||||
<text x="400" y="213" text-anchor="middle" font-size="10" fill="#60a5fa">Circuit breaker · MCP server (37 tools) · A2A · Memory · Guardrails</text>
|
||||
<text x="400" y="196" text-anchor="middle" font-size="10" fill="#60a5fa">RTK compression · Caveman terse-mode · Auto-Combo (19 strategies)</text>
|
||||
<text x="400" y="213" text-anchor="middle" font-size="10" fill="#60a5fa">Circuit breaker · MCP server (110 tools) · A2A · Memory · Guardrails</text>
|
||||
|
||||
<!-- Arrows from router to tiers -->
|
||||
<line x1="310" y1="224" x2="150" y2="285" stroke="#9ca3af" stroke-width="1.5" marker-end="url(#arrow-dark)"/>
|
||||
|
||||
|
Before Width: | Height: | Size: 4.8 KiB After Width: | Height: | Size: 4.8 KiB |
@@ -13,7 +13,7 @@
|
||||
|
||||
<!-- Title -->
|
||||
<text x="400" y="32" text-anchor="middle" font-size="18" font-weight="700" fill="#111827">OmniRoute 3-tier fallback</text>
|
||||
<text x="400" y="52" text-anchor="middle" font-size="12" fill="#6b7280">Never stop building — automatic zero-config failover across 207+ providers</text>
|
||||
<text x="400" y="52" text-anchor="middle" font-size="12" fill="#6b7280">Never stop building — automatic zero-config failover across 352 providers</text>
|
||||
|
||||
<!-- Client box -->
|
||||
<rect x="275" y="70" width="250" height="52" rx="8" fill="#ffffff" stroke="#d1d5db" stroke-width="1.5"/>
|
||||
@@ -27,8 +27,8 @@
|
||||
<!-- Router box -->
|
||||
<rect x="200" y="152" width="400" height="72" rx="8" fill="#dbeafe" stroke="#3b82f6" stroke-width="1.5"/>
|
||||
<text x="400" y="177" text-anchor="middle" font-size="14" font-weight="700" fill="#1d4ed8">OmniRoute Smart Router</text>
|
||||
<text x="400" y="196" text-anchor="middle" font-size="10" fill="#3b82f6">RTK compression · Caveman terse-mode · Auto-Combo (14 strategies)</text>
|
||||
<text x="400" y="213" text-anchor="middle" font-size="10" fill="#3b82f6">Circuit breaker · MCP server (37 tools) · A2A · Memory · Guardrails</text>
|
||||
<text x="400" y="196" text-anchor="middle" font-size="10" fill="#3b82f6">RTK compression · Caveman terse-mode · Auto-Combo (19 strategies)</text>
|
||||
<text x="400" y="213" text-anchor="middle" font-size="10" fill="#3b82f6">Circuit breaker · MCP server (110 tools) · A2A · Memory · Guardrails</text>
|
||||
|
||||
<!-- Arrows from router to tiers -->
|
||||
<line x1="310" y1="224" x2="150" y2="285" stroke="#6b7280" stroke-width="1.5" marker-end="url(#arrow-light)"/>
|
||||
|
||||
|
Before Width: | Height: | Size: 4.8 KiB After Width: | Height: | Size: 4.8 KiB |
122
scripts/ad-hoc/gen-openapi-missing-paths.mjs
Normal file
122
scripts/ad-hoc/gen-openapi-missing-paths.mjs
Normal file
@@ -0,0 +1,122 @@
|
||||
#!/usr/bin/env node
|
||||
// One-shot generator (2026-08-31 docs audit follow-up nº 3): append a minimal,
|
||||
// honest OpenAPI entry for every real route that docs/openapi.yaml does not
|
||||
// document yet. Enumerates routes with the SAME lib the check:api-docs-refs
|
||||
// gate uses, so the generated set can never diverge from the gate's universe.
|
||||
// Minimal by design: real methods (parsed from each route.ts's exports), a
|
||||
// group tag, a neutral path-derived summary and a generic 200 — no invented
|
||||
// semantics. Rich schemas stay hand-curated in the existing entries.
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { collectApiRouteFiles, toApiUrlPath, apiRoot } from "../check/lib/apiRoutes.mjs";
|
||||
|
||||
const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..", "..");
|
||||
const SPEC = path.join(ROOT, "docs", "openapi.yaml");
|
||||
const APPLY = process.argv.includes("--apply");
|
||||
|
||||
const normalizeParams = (p) => p.replace(/\{[^}]+\}/g, "{}");
|
||||
|
||||
// --- real routes + their exported HTTP methods --------------------------------
|
||||
const METHOD_RE =
|
||||
/export\s+(?:async\s+)?function\s+(GET|POST|PUT|PATCH|DELETE|HEAD|OPTIONS)\b|export\s+const\s+(GET|POST|PUT|PATCH|DELETE|HEAD|OPTIONS)\b|export\s*\{[^}]*\b(GET|POST|PUT|PATCH|DELETE|HEAD|OPTIONS)\b[^}]*\}/g;
|
||||
|
||||
function routeMethods(absFile) {
|
||||
const src = fs.readFileSync(absFile, "utf8");
|
||||
const methods = new Set();
|
||||
for (const m of src.matchAll(METHOD_RE)) {
|
||||
const name = m[1] || m[2];
|
||||
if (name) methods.add(name);
|
||||
if (m[3]) {
|
||||
// re-export list: capture every method inside the braces
|
||||
for (const inner of m[0].matchAll(/\b(GET|POST|PUT|PATCH|DELETE|HEAD|OPTIONS)\b/g))
|
||||
methods.add(inner[1]);
|
||||
}
|
||||
}
|
||||
methods.delete("OPTIONS"); // CORS preflight — not a documented operation
|
||||
methods.delete("HEAD");
|
||||
return [...methods];
|
||||
}
|
||||
|
||||
const routeFiles = collectApiRouteFiles(ROOT);
|
||||
const API_ROOT = apiRoot(ROOT);
|
||||
const routes = new Map(); // urlPath -> methods
|
||||
for (const rel of routeFiles) {
|
||||
const abs = path.join(ROOT, rel);
|
||||
const url = toApiUrlPath(path.dirname(abs), API_ROOT);
|
||||
if (url) routes.set(url, routeMethods(abs));
|
||||
}
|
||||
|
||||
// --- paths already in the spec -------------------------------------------------
|
||||
const spec = fs.readFileSync(SPEC, "utf8");
|
||||
const specPaths = new Set();
|
||||
for (const m of spec.matchAll(/^ {2}(\/[^\s:]+):\s*$/gm)) specPaths.add(normalizeParams(m[1]));
|
||||
|
||||
const missing = [...routes.entries()]
|
||||
.filter(([url]) => !specPaths.has(normalizeParams(url)))
|
||||
.filter(([, methods]) => methods.length > 0)
|
||||
.sort(([a], [b]) => a.localeCompare(b));
|
||||
|
||||
// --- tag + summary derivation --------------------------------------------------
|
||||
const cap = (s) => s.charAt(0).toUpperCase() + s.slice(1);
|
||||
function groupTag(url) {
|
||||
const seg = url.replace(/^\/api\//, "").split("/");
|
||||
if (seg[0] === "v1") return seg[1] ? `V1 ${cap(seg[1].replace(/\{|\}/g, ""))}` : "V1";
|
||||
return cap(seg[0].replace(/\{|\}/g, "").replace(/-/g, " "));
|
||||
}
|
||||
function summaryFor(url, method) {
|
||||
const tail = url
|
||||
.replace(/^\/api\/(v1\/)?/, "")
|
||||
.replace(/\{([^}]+)\}/g, "<$1>")
|
||||
.replace(/[/]/g, " › ")
|
||||
.replace(/-/g, " ");
|
||||
return `${method} ${tail}`;
|
||||
}
|
||||
|
||||
// --- emit YAML -----------------------------------------------------------------
|
||||
const existingTags = new Set(
|
||||
[...spec.matchAll(/^ {2}- name: (.+)$/gm)].map((m) => m[1].trim().toLowerCase())
|
||||
);
|
||||
const newTags = new Map();
|
||||
const lines = [];
|
||||
lines.push("");
|
||||
lines.push(" # --- Generated route coverage (docs audit 2026-08-31) -----------------------");
|
||||
lines.push(" # Minimal entries for every implemented route not documented above. Methods");
|
||||
lines.push(" # are parsed from each route.ts's exports; summaries are path-derived.");
|
||||
lines.push(" # Regenerate with: node scripts/ad-hoc/gen-openapi-missing-paths.mjs --apply");
|
||||
for (const [url, methods] of missing) {
|
||||
const tag = groupTag(url);
|
||||
if (!existingTags.has(tag.toLowerCase()) && !newTags.has(tag))
|
||||
newTags.set(tag, `${tag} endpoints (generated route coverage)`);
|
||||
lines.push(` ${url}:`);
|
||||
for (const method of methods.sort()) {
|
||||
lines.push(` ${method.toLowerCase()}:`);
|
||||
lines.push(` tags:`);
|
||||
lines.push(` - ${tag}`);
|
||||
lines.push(` summary: "${summaryFor(url, method)}"`);
|
||||
lines.push(` responses:`);
|
||||
lines.push(` "200":`);
|
||||
lines.push(` description: OK`);
|
||||
}
|
||||
}
|
||||
|
||||
const tagLines = [...newTags.entries()]
|
||||
.sort(([a], [b]) => a.localeCompare(b))
|
||||
.map(([name, description]) => ` - name: ${name}\n description: ${description}`)
|
||||
.join("\n");
|
||||
|
||||
console.log(
|
||||
`real routes: ${routes.size} · already in spec: ${specPaths.size} · missing with methods: ${missing.length} · new tags: ${newTags.size}`
|
||||
);
|
||||
if (!APPLY) {
|
||||
console.log("(dry-run) pass --apply to write docs/openapi.yaml");
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
let out = spec;
|
||||
// append new tags right after the last existing tag entry (before `paths:`)
|
||||
if (tagLines) out = out.replace(/\npaths:\n/, `\n${tagLines}\n\npaths:\n`);
|
||||
// insert generated paths right before the components section
|
||||
out = out.replace(/\ncomponents:\n/, `\n${lines.join("\n")}\n\ncomponents:\n`);
|
||||
fs.writeFileSync(SPEC, out);
|
||||
console.log(`wrote ${missing.length} paths + ${newTags.size} tags to docs/openapi.yaml`);
|
||||
@@ -200,7 +200,7 @@ function readCodeFacts() {
|
||||
"const FOREVER=new Set(['recurring-monthly','recurring-daily','recurring-uncapped',",
|
||||
"'recurring-credit','keyless']);",
|
||||
"const ff=new Set();for(const m of t.perModel)if(FOREVER.has(m.freeType))ff.add(m.provider);",
|
||||
'console.log("@@"+JSON.stringify({freeSteady:t.steadyRecurringTokens,',
|
||||
'console.log("@@"+JSON.stringify({freeSteady:t.steadyRecurringTokens,entries:t.perModel.length,',
|
||||
"freeFirst:t.firstMonthRealisticTokens,freePools:t.poolCount,engines:ENGINE_IDS.length,",
|
||||
"cliTotal:cli.length,cliCode:by('code'),cliAgent:by('agent'),",
|
||||
"mcpTools:countUniqueMcpTools(cols),mcpScopes:sc.size,providers:pids.size,freeForever:ff.size}));",
|
||||
@@ -264,6 +264,42 @@ export function checkFreeTierHeadline(content, totals) {
|
||||
};
|
||||
}
|
||||
|
||||
// PURE: docs prose that names the product version ("OmniRoute v3.8.50 ·",
|
||||
// "**Current version:** 3.8.50") must match package.json exactly.
|
||||
export function readPackageVersion() {
|
||||
try {
|
||||
return String(JSON.parse(fs.readFileSync(path.join(ROOT, "package.json"), "utf8")).version);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
export function makeVersionClaimValidator(expected) {
|
||||
const PATTERNS = [
|
||||
/OmniRoute v(\d+\.\d+\.\d+)/g,
|
||||
/Current version:\*{0,2}\s*\*{0,2}(\d+\.\d+\.\d+)/g,
|
||||
];
|
||||
return (content) => {
|
||||
if (!expected) return { ok: false, detail: "package.json version could not be read" };
|
||||
const claims = [];
|
||||
for (const pattern of PATTERNS)
|
||||
for (const m of content.matchAll(pattern)) claims.push({ value: m[1], text: m[0].trim() });
|
||||
if (!claims.length) return { ok: true, detail: "no version claim in this file" };
|
||||
const stale = claims.filter((c) => c.value !== expected);
|
||||
if (!stale.length)
|
||||
return {
|
||||
ok: true,
|
||||
detail: `${claims.length} version claim(s) match package.json ${expected}`,
|
||||
};
|
||||
return {
|
||||
ok: false,
|
||||
detail:
|
||||
`stale version: ${[...new Set(stale.map((c) => `"${c.text}"`))].join(", ")} — ` +
|
||||
`package.json is ${expected}`,
|
||||
};
|
||||
};
|
||||
}
|
||||
|
||||
// --- Generic numeric-claim gate ---------------------------------------------
|
||||
// Same principle as the free-tier headline: docs legitimately carry numbers that are
|
||||
// NOT the aggregate being gated (per-module tool counts like "Memory tool definitions
|
||||
@@ -371,6 +407,8 @@ const SVG_DIAGRAM_FILES = [
|
||||
"docs/diagrams/comparison-table.svg",
|
||||
"docs/diagrams/cli-terminal.svg",
|
||||
"docs/diagrams/tier-cascade.svg",
|
||||
"public/images/tier-flow-dark.svg",
|
||||
"public/images/tier-flow-light.svg",
|
||||
];
|
||||
|
||||
export function buildChecks() {
|
||||
@@ -398,9 +436,20 @@ export function buildChecks() {
|
||||
files: ["README.md", "AGENTS.md", "llm.txt"],
|
||||
validate: makeNumberClaimValidator(countMigrations(), {
|
||||
what: "migrations",
|
||||
pattern: /(\d+)\+? migrations?\b/gi,
|
||||
pattern: /(\d+)\+? (?:versioned )?(?:SQL )?migrations?\b/gi,
|
||||
}),
|
||||
},
|
||||
{
|
||||
// The README footer and llm.txt each carry the product version as prose; both
|
||||
// shipped stale ("v3.8.50" on a 3.8.51 tree) in the 2026-08-31 audit. Compare
|
||||
// every such claim against package.json, which is authoritative.
|
||||
label: "Package version (docs prose)",
|
||||
actual: readPackageVersion(),
|
||||
docKey: "package version",
|
||||
strict: true,
|
||||
files: ["README.md", "llm.txt"],
|
||||
validate: makeVersionClaimValidator(readPackageVersion()),
|
||||
},
|
||||
{
|
||||
label: "i18n locales count",
|
||||
actual: countLocales(),
|
||||
@@ -472,23 +521,55 @@ export function buildChecks() {
|
||||
f.mcpTools,
|
||||
"MCP tools",
|
||||
{
|
||||
pattern: /(\d+) tools/gi,
|
||||
pattern: /(\d+)[- ]tools?\b/gi,
|
||||
// per-module rows ("Memory tool definitions (3 tools)") and the CLI catalog
|
||||
// total ("33 tools (25 CLI Code's …)") are not the MCP aggregate
|
||||
// per-module rows read "… tool definitions (N tools" / "… management tools
|
||||
// (N tools" — the word tool(s)/definitions sits right before the paren. The
|
||||
// aggregate ("MCP Server (109 tools", "all 109 tools") never does.
|
||||
skipBefore: /(tools?|definitions?)\s*\(\s*$/i,
|
||||
// "Phase 2 tool handlers" is a phase number, not a tool count
|
||||
skipBefore: /(tools?|definitions?)\s*\(\s*$|phase\s+$/i,
|
||||
skipAfter: /^\s*\(\d+ CLI/,
|
||||
},
|
||||
["README.md", "AGENTS.md", "docs/frameworks/MCP-SERVER.md"]
|
||||
[
|
||||
"README.md",
|
||||
"AGENTS.md",
|
||||
"docs/frameworks/MCP-SERVER.md",
|
||||
"llm.txt",
|
||||
"open-sse/mcp-server/README.md",
|
||||
"skills/omni-mcp/SKILL.md",
|
||||
]
|
||||
),
|
||||
claim(f.mcpScopes, "MCP scopes", { pattern: /(\d+) scopes/gi }, ["README.md", "AGENTS.md"]),
|
||||
claim(f.cliTotal, "CLI tools", { pattern: /(\d+) tools(?=\s*\(\d+ CLI)/gi }, ["README.md"]),
|
||||
claim(f.freeForever, "free-forever providers", { pattern: /(\d+) free forever/gi }, [
|
||||
claim(f.mcpScopes, "MCP scopes", { pattern: /(\d+) scopes/gi }, [
|
||||
"README.md",
|
||||
"docs/diagrams/promise-pillars.svg",
|
||||
"AGENTS.md",
|
||||
"llm.txt",
|
||||
"skills/omni-mcp/SKILL.md",
|
||||
]),
|
||||
claim(f.cliTotal, "CLI tools", { pattern: /(\d+) tools(?=\s*\(\d+ CLI)/gi }, ["README.md"]),
|
||||
claim(
|
||||
f.freeForever,
|
||||
"free-forever providers",
|
||||
{ pattern: /(\d+)(?:\s+recurring(?:\/|\s+or\s+)keyless)?\s+free[- ]forever/gi },
|
||||
["README.md", "docs/diagrams/promise-pillars.svg"]
|
||||
),
|
||||
claim(
|
||||
f.entries,
|
||||
"free-tier catalog entries",
|
||||
{ pattern: /(\d+) (?:cataloged |catalogued )?(?:free-tier |catalog )entries\b/gi },
|
||||
["README.md", "docs/diagrams/free-tier-budget.svg"]
|
||||
),
|
||||
claim(
|
||||
f.freePools,
|
||||
"recurring pools",
|
||||
{
|
||||
pattern: /(\d+) (?:documented )?(?:recurring|free-tier) pool(?:s|(?:\s+keys))?\b/gi,
|
||||
// "20 recurring pools with a published positive monthly budget" is the
|
||||
// positive-budget SUBSET, not the recurring-pool total — never gate it.
|
||||
skipAfter: /^\s+with a published positive/i,
|
||||
},
|
||||
["README.md", "docs/diagrams/free-tier-budget.svg", "docs/reference/FREE_TIERS.md"]
|
||||
),
|
||||
];
|
||||
})(),
|
||||
{
|
||||
@@ -518,6 +599,7 @@ export function buildChecks() {
|
||||
"docs/guides/FREE_PROVIDER_RANKINGS.md",
|
||||
"docs/diagrams/strategies-grid.svg",
|
||||
"docs/diagrams/auto-combo-12factor.mmd",
|
||||
"llm.txt",
|
||||
"open-sse/services/autoCombo/routerStrategy.ts",
|
||||
"open-sse/services/taskAwareRouter.ts",
|
||||
"tests/unit/lkgp-enabled-context-11181.test.ts",
|
||||
@@ -533,7 +615,7 @@ export function buildChecks() {
|
||||
actual: countRoutingStrategies(),
|
||||
docKey: "strategies",
|
||||
strict: false,
|
||||
files: ["docs/routing/AUTO-COMBO.md", "docs/architecture/RESILIENCE_GUIDE.md"],
|
||||
files: ["docs/routing/AUTO-COMBO.md", "docs/architecture/RESILIENCE_GUIDE.md", "llm.txt"],
|
||||
},
|
||||
{
|
||||
label: "OAuth providers count",
|
||||
|
||||
@@ -27,9 +27,9 @@ Each manifest URL follows the pattern:
|
||||
| ID | Name | Description |
|
||||
| ---------------------- | ----------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `omni-auth` | Authentication | Manage API key authentication and session tokens. Start here to authenticate requests via Bearer token, obtain session cookies, and configure login requirements. |
|
||||
| `omni-providers` | Providers | Manage provider connections, API keys, OAuth flows, and connection tests across OmniRoute's 327-provider catalog. |
|
||||
| `omni-providers` | Providers | Manage provider connections, API keys, OAuth flows, and connection tests across OmniRoute's 327-provider catalog. |
|
||||
| `omni-models` | Models | Query available AI models across all configured providers. List models, resolve model aliases, and browse the full model catalog including provider-specific variants. |
|
||||
| `omni-combos-routing` | Combos & Routing | Create and manage routing combos with 19 strategies (priority, weighted, round-robin, Auto-combo, and more). Configure fallback chains, test routing outcomes, and retrieve combo metrics. |
|
||||
| `omni-combos-routing` | Combos & Routing | Create and manage routing combos with 19 strategies (priority, weighted, round-robin, Auto-combo, and more). Configure fallback chains, test routing outcomes, and retrieve combo metrics. |
|
||||
| `omni-api-keys` | API Keys | Create, list, rotate, and revoke OmniRoute API keys. Control per-key scopes, spending limits, and expiration. |
|
||||
| `omni-usage-logs` | Usage & Logs | Access detailed call logs and usage analytics. Filter by provider, model, time range, status, and cost. Export logs and aggregate token usage. |
|
||||
| `omni-budget` | Budget & Rate Limits | Configure spending limits, token quotas, and rate-limit policies per API key or globally. Inspect current consumption and enforce cost controls. |
|
||||
@@ -44,7 +44,7 @@ Each manifest URL follows the pattern:
|
||||
| `omni-sync-cloud` | Cloud Sync | Synchronise OmniRoute configuration, provider connections, and settings to/from cloud storage. Manage cloud worker authentication and remote backup targets. |
|
||||
| `omni-db-backups` | Database & Backups | Trigger system backups, restore from backup files, and manage the SQLite database lifecycle. Supports export, import, and incremental snapshot strategies. |
|
||||
| `omni-webhooks` | Webhooks | Register, list, test, and remove webhook endpoints. Configure event subscriptions (request.completed, provider.error, budget.exceeded, etc.) and manage delivery retries. |
|
||||
| `omni-mcp` | MCP Server | Connect to the OmniRoute MCP server (107 tools, 3 transports: SSE/stdio/HTTP) across 32 permission scopes. |
|
||||
| `omni-mcp` | MCP Server | Connect to the OmniRoute MCP server (110 tools, 3 transports: SSE/stdio/HTTP) across 33 permission scopes. |
|
||||
| `omni-agents-a2a` | Agents & A2A Protocol | Interact with OmniRoute via JSON-RPC 2.0 agent-to-agent protocol. 6 built-in A2A skills: smart-routing, quota-management, provider-discovery, cost-analysis, health-report, list-capabilities. |
|
||||
| `omni-version-manager` | Version Manager | Install, start, stop, restart, and update embedded services (9Router, CLIProxyAPI). Monitor service status, retrieve logs, and configure auto-start. |
|
||||
| `omni-inference` | Inference (OpenAI-compatible) | The core OpenAI-compatible inference endpoints: chat completions, embeddings, images, audio (TTS/STT), moderations, rerank, and the Responses API. |
|
||||
@@ -80,9 +80,9 @@ Each manifest URL follows the pattern:
|
||||
|
||||
## Configuration workflow (1)
|
||||
|
||||
| ID | Name | Description |
|
||||
| ------------------ | ----------------- | ------------------------------------------------------------------------- |
|
||||
| `config-codex-cli` | Config: Codex CLI | Configure Codex CLI to use OmniRoute as an OpenAI-compatible backend. |
|
||||
| ID | Name | Description |
|
||||
| ------------------ | ----------------- | --------------------------------------------------------------------- |
|
||||
| `config-codex-cli` | Config: Codex CLI | Configure Codex CLI to use OmniRoute as an OpenAI-compatible backend. |
|
||||
|
||||
---
|
||||
|
||||
@@ -111,7 +111,7 @@ examples, and error codes. Assume the reader is an agent with no prior context.
|
||||
|
||||
## What makes OmniRoute skills unique
|
||||
|
||||
- `omni-mcp` — 107 MCP tools across 32 scopes over SSE/stdio/Streamable HTTP
|
||||
- `omni-mcp` — 110 MCP tools across 33 scopes over SSE/stdio/Streamable HTTP
|
||||
- `omni-agents-a2a` — 6 A2A skills (smart-routing, quota, discovery, cost, health, list-capabilities) via JSON-RPC 2.0
|
||||
- `omni-combos-routing` — create/configure combos, 19 strategies, Auto-combo scoring, fallback chains
|
||||
- `omni-compression` — RTK + Caveman + stacked mode + MCP accessibility filter (60–90% token savings)
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user