pjandro/OmniRoute
1
0
Fork 0
mirror of https://github.com/diegosouzapw/OmniRoute.git synced 2026-07-26 09:52:11 +03:00
Code Issues Packages Projects Releases Wiki Activity

docs(wiki): full content parity with docs at v3.8.26 (226 providers / 87 MCP tools / 60 executors / 16 OAuth / 15 strategies / 42 locales)

diegosouzapw
2026-06-15 12:28:57 -03:00
parent 5331dc891a
commit 034815eb26
40 changed files with 4074 additions and 1190 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
A2A-Server.md

@@ -136,18 +136,32 @@ curl -X POST http://localhost:20128/a2a \
## Available Skills
OmniRoute exposes 5 A2A skills wired in `src/lib/a2a/taskExecution.ts::A2A_SKILL_HANDLERS`. Each skill module lives in `src/lib/a2a/skills/`.
OmniRoute exposes 6 A2A skills wired in `src/lib/a2a/taskExecution.ts::A2A_SKILL_HANDLERS`. Each skill module lives in `src/lib/a2a/skills/`.
| Skill | ID | Description |
| :----------------- | :------------------- | :------------------------------------------------------------------------------------------ |
| Smart Routing | `smart-routing` | Routes a prompt through the optimal provider/combo using OmniRoute's combo engine + scoring |
| Quota Management | `quota-management` | Reports per-provider quota state, helps callers decide when to throttle/switch |
| Provider Discovery | `provider-discovery` | Lists installed providers with capabilities, free-tier flags, OAuth status |
| Cost Analysis | `cost-analysis` | Estimates cost of a request/conversation given the catalog + recent usage |
| Health Report | `health-report` | Aggregates circuit breaker, cooldown, lockout state per provider |
| Skill | ID | Description | Tags | Examples |
| :------------------ | :-------------------- | :------------------------------------------------------------------------------------------------------------ | :---------------------- | :-------------------------------------- |
| Smart Routing | `smart-routing` | Routes a prompt through the optimal provider/combo using OmniRoute's combo engine + scoring | routing, providers | "Route this prompt via the best model" |
| Quota Management | `quota-management` | Reports per-provider quota state, helps callers decide when to throttle/switch | quota, providers | "Check quota for anthropic" |
| Provider Discovery | `provider-discovery` | Lists installed providers with capabilities, free-tier flags, OAuth status | providers, discovery | "What providers are available?" |
| Cost Analysis | `cost-analysis` | Estimates cost of a request/conversation given the catalog + recent usage | cost, usage | "Estimate cost for this conversation" |
| Health Report | `health-report` | Aggregates circuit breaker, cooldown, lockout state per provider | health, resilience | "Show health status of all providers" |
| List Capabilities | `list-capabilities` | Returns the full 42-entry Agent Skills catalog as a markdown table with raw SKILL.md URLs for context injection | catalog, discovery, skills | "List all OmniRoute capabilities" |
> Note: the Agent Card description currently advertises "36+ providers" (`src/app/.well-known/agent.json/route.ts:26` and `:55`). The actual catalog has grown to 180+ providers — the string should be updated in a follow-up change (tracked as a separate doc/code TODO; not modified here).
### `list-capabilities` Skill Detail
The `list-capabilities` skill is particularly useful for external agents that need to discover what OmniRoute exposes before sending API calls. It returns a structured markdown table artifact:
```
| ID | Name | Category | Area | Endpoints/Commands | Raw URL |
| --- | --- | --- | --- | --- | --- |
| omni-auth | Auth & Sessions | api | auth | POST /api/auth/login, ... | https://raw.githubusercontent.com/... |
...
```
Each row includes the `rawUrl` column so agents can immediately fetch the full SKILL.md. The `metadata.totalSkills` field is always `42`. Implementation: `src/lib/a2a/skills/listCapabilities.ts`. See also [AGENT-SKILLS.md](./AGENT-SKILLS.md).
---
## REST API (auxiliary)

2
ACP.md

@@ -547,7 +547,7 @@ const agents = detectInstalledAgents();
## What's Next?
- **[API Reference](../reference/API_REFERENCE.md)** — REST API endpoints
- **[Provider Reference](../reference/PROVIDER_REFERENCE.md)** — All 223 providers
- **[Provider Reference](../reference/PROVIDER_REFERENCE.md)** — All 226 providers
- **[MCP Server](./MCP-SERVER.md)** — Model Context Protocol integration
- **[A2A Server](./A2A-SERVER.md)** — Agent-to-Agent protocol
- **[Cloud Agent](./CLOUD_AGENT.md)** — Cloud-based agents

479
API-Reference.md

@@ -151,9 +151,17 @@ Authorization: Bearer your-api-key
| GET | `/v1beta/models` | Gemini |
| POST | `/v1beta/models/{...path}` | Gemini generateContent |
| POST | `/v1/api/chat` | Ollama |
| GET | `/api/v1/vscode/{token}/` | OpenAI catalog alias |
| GET | `/api/v1/vscode/{token}/models` | OpenAI models alias |
| POST | `/api/v1/vscode/{token}/chat/completions` | OpenAI tokenized alias |
| POST | `/api/v1/vscode/{token}/responses` | OpenAI Responses tokenized alias |
| POST | `/api/v1/vscode/{token}/api/chat` | Ollama tokenized alias |
| GET | `/api/v1/vscode/{token}/api/tags` | Ollama tags tokenized alias |
All POST routes follow the same shape: `Bearer your-api-key` + Zod-validated JSON body (`v1RerankSchema`, `v1ModerationSchema`, `v1AudioSpeechSchema`, etc., see `src/shared/validation/schemas.ts`). 4xx is returned on schema failure.
For clients that cannot attach `Authorization: Bearer ...`, OmniRoute also accepts API keys in the URL via either query-string compatibility (`?token=...`, `?apiKey=...`, `?api_key=...`, `?key=...`) or the dedicated `/api/v1/vscode/{token}/...` endpoints documented below.
```bash
# Rerank
POST /v1/rerank { "model": "cohere/rerank-3", "query": "...", "documents": ["..."] }
@@ -240,6 +248,67 @@ Validates a WebSocket upgrade handshake and returns the wire protocol example me
**Auth:** Bearer API key during handshake.
### Responses API over WebSocket (codex only)
```bash
# Same host:port as the HTTP API (default 20128); upgrade the connection:
wscat -c "ws://localhost:20128/v1/responses?api_key=<OMNIROUTE_API_KEY>"
# (or: -H "Authorization: Bearer <OMNIROUTE_API_KEY>")
# First frame MUST be response.create:
{ "type": "response.create", "model": "gpt-5.5", "input": [ { "role": "user", "content": "hi" } ] }
```
A Responses-API-over-WebSocket proxy is wired **exclusively to `codex`** (ChatGPT
backend). It listens on the same port as the API/dashboard at paths `/v1/responses`,
`/responses`, and `/api/v1/responses`. On the first `response.create` frame it
authenticates + prepares via the internal `codex-responses-ws` bridge, selects a
codex OAuth connection, and tunnels to `wss://chatgpt.com/backend-api/codex/responses`
via the `wreq-js` transport. **Non-codex models are rejected** (`codex_ws_provider_required`).
For quota-share routing use `model: "qtSd/<group>/codex/<model>"`. Implemented in
`app/server-ws.mjs` + `scripts/dev/responses-ws-proxy.mjs` + `src/app/api/internal/codex-responses-ws/route.ts`.
**Auth:** Bearer API key during handshake. The bundled HTTP server (`server-ws.mjs`)
must be the active entrypoint (it is, by default, when `app/server-ws.mjs` exists).
#### Model id: use the bare ChatGPT id (no `codex/` prefix)
The OpenAI **Codex CLI** validates the model name client-side when
`supports_websockets = true` and **rejects provider-prefixed ids** like
`codex/gpt-5.5` (`The 'codex/gpt-5.5' model is not supported when using Codex with
a ChatGPT account`). Send the **bare** id (e.g. `gpt-5.5`). OmniRoute's bridge is
codex-only, so it re-resolves a bare id as a codex model
(`resolveCodexWsModelInfo`) before tunneling upstream — even though a bare
`gpt-5.5` would otherwise route to another provider over HTTP.
#### Configuring the OpenAI Codex CLI
Point the Codex CLI at OmniRoute by adding a custom provider with WebSocket
support to `~/.codex/config.toml` (use a separate `CODEX_HOME` to avoid touching
an existing config):
```toml
model = "gpt-5.5" # bare id — NOT "codex/gpt-5.5"
model_provider = "omniroute"
[model_providers.omniroute]
name = "OmniRoute (WS)"
base_url = "http://localhost:20128/v1" # no trailing slash; the WS URL is derived (use https/wss in production)
wire_api = "responses" # only supported value since Feb 2026
supports_websockets = true # enables the Responses-over-WS transport
env_key = "OMNIROUTE_API_KEY" # holds the OmniRoute API key (Bearer)
```
```bash
export OMNIROUTE_API_KEY=sk-... # an OmniRoute API key (any key if REQUIRE_API_KEY=false)
codex exec "Responda apenas: PONG"
```
The CLI upgrades `base_url + /responses` to a WebSocket and OmniRoute tunnels it
to the selected codex OAuth connection. Validated end-to-end against the local
server: ChatGPT returns `codex.rate_limits` + `response.created` and streams the
completion.
---
## Quotas & Issues Reporting
@@ -322,24 +391,26 @@ Response example:
### Usage & Analytics
| Endpoint | Method | Description |
| --------------------------- | ------ | -------------------- |
| `/api/usage/history` | GET | Usage history |
| `/api/usage/logs` | GET | Usage logs |
| `/api/usage/request-logs` | GET | Request-level logs |
| `/api/usage/[connectionId]` | GET | Per-connection usage |
| Endpoint | Method | Description |
| --------------------------- | ----------------- | --------------------------------- |
| `/api/usage/history` | GET | Usage history |
| `/api/usage/logs` | GET | Usage logs |
| `/api/usage/request-logs` | GET | Request-level logs |
| `/api/usage/[connectionId]` | GET | Per-connection usage |
| `/api/usage/token-limits` | GET/POST/DELETE | Per-API-key token-limit budgets |
### Settings
| Endpoint | Method | Description |
| ------------------------------- | ------------- | ------------------------- |
| `/api/settings` | GET/PUT/PATCH | General settings |
| `/api/settings/proxy` | GET/PUT | Network proxy config |
| `/api/settings/proxy/test` | POST | Test proxy connection |
| `/api/settings/ip-filter` | GET/PUT | IP allowlist/blocklist |
| `/api/settings/thinking-budget` | GET/PUT | Reasoning token budget |
| `/api/settings/system-prompt` | GET/PUT | Global system prompt |
| `/api/settings/compression` | GET/PUT | Global compression config |
| Endpoint | Method | Description |
| ------------------------------------- | ------------- | --------------------------------------------------- |
| `/api/settings` | GET/PUT/PATCH | General settings |
| `/api/settings/proxy` | GET/PUT | Network proxy config |
| `/api/settings/proxy/test` | POST | Test proxy connection |
| `/api/settings/ip-filter` | GET/PUT | IP allowlist/blocklist |
| `/api/settings/thinking-budget` | GET/PUT | Reasoning token budget |
| `/api/settings/system-prompt` | GET/PUT | Global system prompt |
| `/api/settings/compression` | GET/PUT | Global compression config |
| `/api/settings/purge-request-history` | POST | Clear request log rows and local call-log artifacts |
### Context & Compression
@@ -466,7 +537,6 @@ These endpoints mirror Gemini's API format for clients that expect native Gemini
| `/api/restart` | POST | Trigger graceful server restart |
| `/api/shutdown` | POST | Trigger graceful server shutdown |
| `/api/system/env/repair` | POST | Repair OAuth provider environment variables |
| `/api/system-info` | GET | Generate system diagnostics report |
> **Note:** These endpoints are used internally by the system or for Ollama client compatibility. They are not typically called by end users.
@@ -543,6 +613,39 @@ GET /api/tags
Requests are automatically translated between Ollama and internal formats.
## Tokenized VS Code / Headerless Aliases
Use these aliases when an integration cannot inject an `Authorization` header and needs the API key embedded in the base URL.
```bash
# OpenAI-style catalog alias
GET /api/v1/vscode/{token}/
GET /api/v1/vscode/{token}/models
# OpenAI-style chat aliases
POST /api/v1/vscode/{token}/chat/completions
POST /api/v1/vscode/{token}/responses
# Ollama-style aliases
POST /api/v1/vscode/{token}/api/chat
GET /api/v1/vscode/{token}/api/tags
```
Example:
```bash
curl https://your-host.example/api/v1/vscode/YOUR_API_KEY/models
curl -X POST https://your-host.example/api/v1/vscode/YOUR_API_KEY/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"auto","messages":[{"role":"user","content":"hello"}]}'
```
Notes:
- The tokenized aliases reuse the same handlers as `/v1/*` and `/api/tags`; response shapes stay identical.
- Prefer `Authorization: Bearer ...` whenever the client supports custom headers.
- URL-based tokens may appear in reverse-proxy logs, browser history, and telemetry outside OmniRoute. Treat them as a compatibility option, not the default authentication mode.
---
## Telemetry
@@ -587,6 +690,33 @@ Content-Type: application/json
> **Schema notes** (`setBudgetSchema`): `apiKeyId` is required; at least one of `dailyLimitUsd`, `weeklyLimitUsd`, or `monthlyLimitUsd` must be greater than zero. Optional fields: `warningThreshold` (0–1), `resetInterval` (`daily` | `weekly` | `monthly`), `resetTime` (`HH:MM`). The legacy `{keyId, limit, period}` shape returns `400 Bad Request`.
## Token Limits
Per-API-key **token** budgets (distinct from the USD-based Budget above). Enforced inline on the request path: when a key's current window usage reaches its limit, requests are rejected with `429 Too Many Requests`. Limits can be scoped to a specific `model`, a `provider`, or applied `global`ly across the key; when several limits match a request, the most restrictive one wins.
```bash
# List a key's token limits (includes live window usage)
GET /api/usage/token-limits?apiKeyId=key-123
# Create or update a token limit
POST /api/usage/token-limits
Content-Type: application/json
{
"apiKeyId": "key-123",
"scopeType": "model",
"scopeValue": "openai/gpt-4o",
"tokenLimit": 1000000,
"resetInterval": "monthly",
"enabled": true
}
# Delete a token limit by id
DELETE /api/usage/token-limits?id=tl-abc
```
> **Schema notes** (`setTokenLimitSchema`): `apiKeyId` and `scopeType` (`model` | `provider` | `global`) are required. `scopeValue` is required unless `scopeType` is `global` (e.g. a model id for `model` scope, a provider id for `provider` scope). `tokenLimit` must be a positive integer (coerced from string). Optional: `id` (omit to create, supply to update), `resetInterval` (`daily` | `weekly` | `monthly`, default `monthly`), `resetTime` (`HH:MM`), `enabled` (default `true`). `GET` responses enrich each limit with `tokensUsed`, `remaining`, `windowStart`, `periodStartAt`, and `nextResetAt`. This is a management-class endpoint (auth enforced centrally by the authz pipeline).
## Request Processing
1. Client sends request to `/v1/*`
@@ -709,6 +839,8 @@ OmniRoute exposes three independent temporary-failure mechanisms; the management
| Connection cooldown | `rateLimitedUntil` on provider connections | `/api/rate-limits`, `/api/providers/[id]` | (re-enables lazily; clear via provider PUT) |
| Model lockout | In-memory model-availability registry | `GET /api/resilience/model-cooldowns` | `DELETE /api/resilience/model-cooldowns` |
`PATCH /api/resilience` accepts provider breaker overrides under `providerBreaker.oauth` and `providerBreaker.apikey`. Each profile supports `degradationThreshold`, `failureThreshold`, and `resetTimeoutMs`; the same fields are exposed in Dashboard → Settings → Resilience.
```bash
# Clear a single model lockout
curl -X DELETE http://localhost:20128/api/resilience/model-cooldowns \
@@ -859,6 +991,321 @@ Returns the public A2A agent card (name, description, capabilities, skill catalo
---
## ACP (Agent Client Protocol) Management
The ACP framework lets you spawn CLI agents (Claude Code, Codex, Gemini CLI, etc.)
as child processes. These endpoints manage ACP agent detection and custom agent
registration.
| Method | Path | Description |
| ------ | ----------------------- | ---------------------------------------------------------------------------------------- |
| GET | `/api/acp/agents` | List all known CLI agents (built-in + custom) with installation status, version, binary |
| POST | `/api/acp/agents` | Register a custom ACP agent or refresh cache — body: `{id, name, binary, versionCommand, providerAlias, spawnArgs, protocol}` or `{action: "refresh"}` |
| DELETE | `/api/acp/agents` | Remove a custom ACP agent — query param: `?id=<agentId>` |
**Response example** (`GET /api/acp/agents`):
```json
{
"agents": [
{
"id": "claude",
"name": "Claude Code CLI",
"binary": "claude",
"version": "1.0.45",
"installed": true,
"protocol": "stdio",
"providerAlias": "claude",
"isCustom": false
},
{
"id": "my-custom-cli",
"name": "My Custom CLI",
"installed": false,
"protocol": "stdio",
"providerAlias": "my-provider",
"isCustom": true
}
],
"cacheTtlMs": 60000,
"cacheAge": 1234
}
```
**Auth:** Requires management session (dashboard `auth_token` cookie) or a
management-scoped API key.
See [ACP Framework](../frameworks/ACP.md) for full details.
---
## Analytics & Observability
Real-time analytics endpoints for monitoring routing, compression, and provider
diversity. These power the `/dashboard/analytics/*` pages.
### Auto-routing analytics
| Method | Path | Description |
| ------ | ----------------------------------- | -------------------------------------------------------------------------------------------- |
| GET | `/api/analytics/auto-routing` | Aggregate auto-routing stats: total calls, strategy distribution, tier distribution, top providers |
| GET | `/api/analytics/auto-routing?days=7` | Time-windowed stats (default 24h) |
**Response example**:
```json
{
"window": "24h",
"totalCalls": 1234,
"strategyBreakdown": {
"rules": 800,
"cost": 200,
"latency": 150,
"sla-aware": 50,
"lkgp": 34
},
"tierBreakdown": {
"ultra": 100,
"pro": 500,
"standard": 400,
"free": 234
},
"topProviders": [
{ "provider": "openai", "calls": 500, "avgLatencyMs": 850 },
{ "provider": "anthropic", "calls": 300, "avgLatencyMs": 1200 }
]
}
```
### Compression analytics
| Method | Path | Description |
| ------ | --------------------------------- | ------------------------------------------------------------------------------------ |
| GET | `/api/analytics/compression` | Aggregate compression stats: tokens saved, savings %, mode distribution, engine usage |
**Response example**:
```json
{
"window": "24h",
"totalOriginalTokens": 5000000,
"totalCompressedTokens": 3500000,
"totalSavings": 1500000,
"savingsPct": 30.0,
"modeBreakdown": {
"lite": 400,
"standard": 600,
"aggressive": 100,
"ultra": 50,
"rtk": 84
},
"engineBreakdown": {
"caveman": 800,
"rtk": 434
}
}
```
### Provider diversity tracking
| Method | Path | Description |
| ------ | --------------------------------- | -------------------------------------------------------------------------------------------- |
| GET | `/api/analytics/diversity` | Shannon entropy-based diversity tracking: prevents single points of failure by measuring provider spread |
**Response example**:
```json
{
"window": "24h",
"shannonEntropy": 2.45,
"maxEntropy": 3.17,
"diversityRatio": 0.77,
"providerUsage": {
"openai": 0.40,
"anthropic": 0.25,
"google": 0.20,
"kiro": 0.15
},
"warnings": [
"OpenAI accounts for 40% of traffic — consider diversifying"
]
}
```
**Auth:** Requires management session or management-scoped API key.
---
## Admin Operations
Admin-only endpoints for operational management.
| Method | Path | Description |
| ------ | ------------------------------- | ---------------------------------------------------------------------------------------------- |
| GET | `/api/admin/concurrency` | Read current concurrency limits (global + per-provider) |
| POST | `/api/admin/concurrency` | Update concurrency limits — body: `{global?: number, perProvider?: Record<string, number>}` |
**Auth:** Requires management session with admin scope.
---
## CLI Tools Management
Manage CLI tools that integrate with OmniRoute (antigravity, chipotle, commandCode,
devin-cli, etc.). See [Provider Reference](./PROVIDER_REFERENCE.md) for the full list.
| Method | Path | Description |
| ------ | --------------------------------- | -------------------------------------------------------------------------------------------- |
| GET | `/api/cli-tools/all-statuses` | Status of all CLI tools (installed, version, last seen) |
| GET | `/api/cli-tools/[id]/status` | Status of a specific CLI tool (id can be: antigravity, chipotle, commandCode, devin-cli, etc.) |
| POST | `/api/cli-tools/apply` | Apply a CLI tool configuration to a provider connection |
| GET | `/api/cli-tools/backups` | List CLI tool configuration backups |
| POST | `/api/cli-tools/backups` | Create a backup of all CLI tool configurations |
| POST | `/api/cli-tools/[id]/restore` | Restore a CLI tool from a backup |
| GET | `/api/cli-tools/antigravity-mitm` | Antigravity MITM proxy status (the "antigravity-mitm" CLI tool) |
| POST | `/api/cli-tools/antigravity-mitm/alias` | Configure antigravity-mitm aliases |
**Auth:** Requires management session.
---
## Agent Skills
Manage AI agent skills (similar to OpenAI's custom GPTs but for agents).
| Method | Path | Description |
| ------ | --------------------------------- | -------------------------------------------------------------------------------------------- |
| GET | `/api/agent-skills` | List all agent skills (built-in + custom) |
| GET | `/api/agent-skills/[id]` | Get a specific agent skill |
| POST | `/api/agent-skills` | Create a custom agent skill — body: `{name, description, prompt, model?, temperature?}` |
| PUT | `/api/agent-skills/[id]` | Update a custom agent skill |
| DELETE | `/api/agent-skills/[id]` | Delete a custom agent skill |
| GET | `/api/agent-skills/[id]/raw` | Get raw prompt + metadata (no execution) |
| POST | `/api/agent-skills/generate` | AI-generate a new skill from a natural language description |
**Auth:** Requires management session or management-scoped API key.
---
## Cache Management
Manage the semantic cache and reasoning cache.
| Method | Path | Description |
| ------ | --------------------------------- | -------------------------------------------------------------------------------------------- |
| GET | `/api/cache` | Cache overview: total entries, hit rate, size on disk |
| GET | `/api/cache/entries` | List cached entries (with pagination) |
| DELETE | `/api/cache/entries` | Delete cache entries (filter by query parameters) |
| GET | `/api/cache/stats` | Detailed cache statistics (per-provider, per-model) |
| GET | `/api/cache/reasoning` | Reasoning cache status (for reasoning replay) |
| DELETE | `/api/cache/reasoning` | Clear reasoning cache — query params: `?toolCallId=<id>` (single) or `?provider=<p>` or no params (all) |
**Auth:** Requires management session.
---
## Memory System
Manage persistent memory (FTS5 + vector embeddings).
| Method | Path | Description |
| ------ | --------------------------------- | -------------------------------------------------------------------------------------------- |
| GET | `/api/memory` | List memory entries (filter by scope, type, search query) |
| POST | `/api/memory` | Create a new memory entry — body: `{scope, type, content, metadata?}` |
| GET | `/api/memory/[id]` | Get a specific memory entry |
| PUT | `/api/memory/[id]` | Update a memory entry |
| DELETE | `/api/memory/[id]` | Delete a memory entry |
| GET | `/api/memory/search` | Search memory (FTS5 + vector) |
| POST | `/api/memory/clear` | Clear memory entries (with filters) |
| GET | `/api/memory/stats` | Memory statistics (total entries, embedding coverage, etc.) |
**Auth:** Requires management session or management-scoped API key.
---
## Webhooks
Manage webhook subscriptions for events.
| Method | Path | Description |
| ------ | --------------------------------- | -------------------------------------------------------------------------------------------- |
| GET | `/api/webhooks` | List all webhook subscriptions |
| POST | `/api/webhooks` | Create a webhook subscription — body: `{url, events[], secret?, active?}` |
| GET | `/api/webhooks/[id]` | Get a specific webhook subscription |
| PUT | `/api/webhooks/[id]` | Update a webhook subscription |
| DELETE | `/api/webhooks/[id]` | Delete a webhook subscription |
| GET | `/api/webhooks/events` | List all available webhook event types |
| GET | `/api/webhooks/[id]/deliveries` | List delivery history for a webhook (success/failure log) |
| POST | `/api/webhooks/[id]/test` | Send a test event to a webhook |
**Auth:** Requires management session.
See [Webhooks Framework](../frameworks/WEBHOOKS.md) for full event types.
---
## Skills Framework
Manage Skills (the agentic extensions framework).
| Method | Path | Description |
| ------ | --------------------------------- | -------------------------------------------------------------------------------------------- |
| GET | `/api/skills` | List all installed skills (built-in + custom) |
| POST | `/api/skills/install` | Install a skill from a local path or URL |
| DELETE | `/api/skills/[id]` | Uninstall a skill |
| PUT | `/api/skills/[id]` | Enable or disable a skill — body: `{enabled?: boolean, mode?: "on" \| "off" \| "auto"}` |
| POST | `/api/skills/executions` | Execute a skill — body: `{skillName, apiKeyId, input?, sessionId?}` |
| GET | `/api/skills/executions` | List execution history for all skills (filter by `?apiKeyId=`) |
**Auth:** Requires management session or management-scoped API key.
See [Skills Framework](../frameworks/SKILLS.md) for full details.
---
## Plugins
Manage OmniRoute plugins (third-party extensions).
| Method | Path | Description |
| ------ | --------------------------------- | -------------------------------------------------------------------------------------------- |
| GET | `/api/plugins` | List installed plugins |
| POST | `/api/plugins/install` | Install a plugin from a local path or URL |
| DELETE | `/api/plugins/[name]` | Uninstall a plugin |
| POST | `/api/plugins/[name]/activate` | Activate a plugin |
| POST | `/api/plugins/[name]/deactivate` | Deactivate a plugin |
| GET | `/api/plugins/[name]/config` | Get plugin configuration |
| PUT | `/api/plugins/[name]/config` | Update plugin configuration |
**Auth:** Requires management session.
See [Plugins Framework](../plugins/PLUGIN_SDK.md) for full details.
---
## Shadow Routing
Shadow / A-B comparison of providers is **not a standalone REST surface** — it is configured through combo routing (see [Auto-Combo](../routing/AUTO-COMBO.md)). Per-combo comparison metrics are served by `GET /api/combos/metrics`.
---
## Guardrails
Inspect the runtime guardrails (PII detection, prompt injection detection, vision bridging). Guardrails run on every request; per-call opt-out is via the `x-omniroute-disabled-guardrails` request header — there is no persisted enable/disable surface.
| Method | Path | Description |
| ------ | ---------------------- | ---------------------------------------------------------------------------------------- |
| GET | `/api/guardrails` | List the registered guardrails and their status (name / enabled / priority) |
| POST | `/api/guardrails/test` | Dry-run the pre-call pipeline over a sample input — body: `{input, disabledGuardrails?}` |
**Auth:** Requires management session.
See [Security > Guardrails](../security/GUARDRAILS.md) for full details.
---
---
## Authentication
- Dashboard routes (`/dashboard/*`) use `auth_token` cookie

2
Agent-Skills.md

@@ -159,7 +159,7 @@ See [A2A-SERVER.md](./A2A-SERVER.md) for protocol details.
| `omni-sync-cloud` | sync-cloud | Cloud sync |
| `omni-db-backups` | db-backups | Database backups |
| `omni-webhooks` | webhooks | Webhook event dispatcher |
| `omni-mcp` | mcp | MCP server (37 tools, 3 transports) |
| `omni-mcp` | mcp | MCP server (87 tools, 3 transports) |
| `omni-agents-a2a` | agents-a2a | A2A agent protocol |
| `omni-version-manager` | version-manager | Version and update management |
| `omni-inference` | inference | Direct inference / completions |

2
Agentbridge.md

@@ -19,7 +19,7 @@ When an IDE agent (e.g., GitHub Copilot, Cursor, Claude Code) makes an API call,
This means you can:
- **Reroute any agent to any provider**: Copilot talking to OpenAI? Redirect it to Anthropic Claude, Gemini, or any of OmniRoute's 160+ providers.
- **Reroute any agent to any provider**: Copilot talking to OpenAI? Redirect it to Anthropic Claude, Gemini, or any of OmniRoute's 226+ providers.
- **Apply model mappings**: `gemini-3-flash` → `claude-sonnet-4.7` transparently at the handler level.
- **Observe all agent traffic**: every intercepted request is published to the [Traffic Inspector](./TRAFFIC_INSPECTOR.md).
- **Apply OmniRoute resilience**: combo routing, circuit breakers, fallbacks, and cost tracking work for IDE agent traffic too.

35
Architecture.md

@@ -14,13 +14,13 @@ It provides a single OpenAI-compatible endpoint (`/v1/*`) and routes traffic acr
Core capabilities:
- OpenAI-compatible API surface for CLI/tools (226 providers, 38 executors)
- OpenAI-compatible API surface for CLI/tools (226 providers, 60 executors)
- Request/response translation across provider formats
- Model combo fallback (multi-model sequence)
- Structured combo steps (`provider + model + connection`) with runtime ordering by `compositeTiers`
- Account-level fallback (multi-account per provider)
- Quota preflight and quota-aware P2C account selection in the main chat path
- OAuth + API-key provider connection management (14 OAuth modules)
- OAuth + API-key provider connection management (16 OAuth modules)
- Embedding generation via `/v1/embeddings` (6 providers, 9 models)
- Image generation via `/v1/images/generations` (10+ providers, 20+ models)
- Audio transcription via `/v1/audio/transcriptions` (7 providers)
@@ -63,7 +63,7 @@ Core capabilities:
- Prompt injection guard middleware
- Prompt compression pipeline with Caveman, RTK, stacked pipelines, compression combos, language packs, and analytics
- ACP (Agent Communication Protocol) registry
- Modular OAuth providers (14 individual modules under `src/lib/oauth/providers/`)
- Modular OAuth providers (16 individual modules under `src/lib/oauth/providers/`)
- Uninstall/full-uninstall scripts
- OAuth environment repair action
- WebSocket bridge for OpenAI-compatible WS clients (`/v1/ws`)
@@ -318,12 +318,35 @@ Domain layer modules:
- Eval runner: `src/lib/domain/evalRunner.ts`
- Domain state persistence: `src/lib/db/domainState.ts` — SQLite CRUD for fallback chains, budgets, cost history, lockout state, circuit breakers
OAuth provider modules (14 individual files under `src/lib/oauth/providers/`):
OAuth provider modules (16 individual files under `src/lib/oauth/providers/`):
- Registry index: `src/lib/oauth/providers/index.ts`
- Individual providers: `claude.ts`, `codex.ts`, `gemini.ts`, `antigravity.ts`, `qoder.ts`, `qwen.ts`, `kimi-coding.ts`, `github.ts`, `kiro.ts`, `cursor.ts`, `kilocode.ts`, `cline.ts`, `windsurf.ts`, `gitlab-duo.ts`
- Individual providers: `claude.ts`, `codex.ts`, `gemini.ts`, `antigravity.ts`, `agy.ts`, `qoder.ts`, `qwen.ts`, `kimi-coding.ts`, `github.ts`, `kiro.ts`, `cursor.ts`, `kilocode.ts`, `cline.ts`, `windsurf.ts`, `gitlab-duo.ts`, `trae.ts`
- Thin wrapper: `src/lib/oauth/providers.ts` — re-exports from individual modules
## 5) Embedded Services (v3.8.4)
OmniRoute can install, supervise, and route to locally-running AI tool processes
called **embedded services**. Two are shipped in v3.8.4: 9Router and CLIProxyAPI.
Architecture layers:
- **UI** (`/dashboard/providers/services`) — two-tab page with lifecycle controls,
live log streaming, API key management, and (for 9Router) embedded native UI via
an internal reverse proxy.
- **API** (`/api/services/{name}/*`) — 8 endpoints for 9Router, 7 for CLIProxyAPI,
all classified **LOCAL_ONLY** (hard rule #17). A shared `GET /api/services/[name]/logs`
SSE endpoint serves both services.
- **Supervisor** (`src/lib/services/`) — generic `ServiceSupervisor` class wraps
`child_process.spawn`, holds a 5 MB ring buffer for SSE log streaming, a health
probe loop, an atomic operation lock, and a SIGTERM→SIGKILL graceful shutdown.
`bootstrap.ts` wires all configured services at process start.
- **Provider/executor** (`open-sse/executors/ninerouter.ts`) — 9Router is exposed as
a real provider. Models are prefixed `9router/{sub}/{model}` and synced every 5 min
from 9Router's `/v1/models` endpoint.
Deep-dive: `docs/frameworks/EMBEDDED-SERVICES.md`
## Major Subsystems (v3.8.0)
### A. Auto Combo Engine
@@ -339,7 +362,7 @@ relying on a static combo definition. It powers the `auto/*` model prefix family
Key capabilities:
- **15 routing strategies** (priority, weighted, fill-first, round-robin, P2C, random,
- **14 routing strategies** (priority, weighted, fill-first, round-robin, P2C, random,
least-used, cost-optimized, strict-random, **auto**, lkgp, context-optimized,
context-relay, plus a fallback path) — auto is the headline addition in v3.8.0.
- **9-factor scoring**: cost, latency p95, success rate, quota headroom, lockout

18
Authz-Guide.md

@@ -4,7 +4,7 @@
# Authorization Guide
> **Source of truth:** `src/server/authz/`, `src/shared/constants/publicApiRoutes.ts`, `src/lib/api/requireManagementAuth.ts`, `src/shared/utils/apiAuth.ts`
> **Last updated:** 2026-05-13 — v3.8.0
> **Last updated:** 2026-06-05 — v3.8.2
OmniRoute has a route-aware authorization pipeline that gates every API request. Classification is **deterministic** and **fail-closed** — anything that cannot be classified ends up as `MANAGEMENT` and demands a session or management-grade token. This page explains the model for engineers maintaining routes or designing new endpoints.
@@ -40,16 +40,16 @@ Some management routes accept **either** mode: cookie OR `Bearer <key>` when the
`src/server/authz/types.ts` defines three classes; any route that cannot be classified deterministically falls back to `MANAGEMENT`.
| Class | Description | Auth required |
| ------------ | ---------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------- |
| `PUBLIC` | Explicitly safe routes — login, logout, status, init, health, onboarding bootstrap. | None |
| `CLIENT_API` | Model-serving endpoints — `/api/v1/*`, plus aliases `/v1/*`, `/chat/completions`, `/responses`, `/models`, `/codex/*`. | Bearer key (unless `REQUIRE_API_KEY != "true"`) |
| `MANAGEMENT` | Dashboard pages, settings, providers, keys, admin and diagnostics endpoints. | Dashboard session OR Bearer with `manage` scope |
| Class | Description | Auth required |
| ------------ | ---------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- |
| `PUBLIC` | Explicitly safe routes — login, logout, status, init, health, onboarding bootstrap. | None |
| `CLIENT_API` | Model-serving endpoints — `/api/v1/*`, plus aliases `/v1/*`, `/chat/completions`, `/responses`, `/models`, `/codex/*`. | Bearer key when the effective `REQUIRE_API_KEY` feature flag is enabled |
| `MANAGEMENT` | Dashboard pages, settings, providers, keys, admin and diagnostics endpoints. | Dashboard session OR Bearer with `manage` scope |
## Pipeline
```
Incoming request → src/middleware.ts
Incoming request → src/proxy.ts
→ runAuthzPipeline() in src/server/authz/pipeline.ts
1. Strip trusted internal headers (x-omniroute-auth-*, x-omniroute-route-class)
2. Generate request id, classify route via classifyRoute()
@@ -70,7 +70,7 @@ Trusted internal headers (defined in `src/server/authz/headers.ts`) are **stripp
Each route class has a policy in `src/server/authz/policies/`:
- **`publicPolicy`** (`policies/public.ts`) — always returns `allow({ kind: "anonymous", id: "anonymous" })`.
- **`clientApiPolicy`** (`policies/clientApi.ts`) — extracts Bearer, validates via `validateApiKey()`. Falls through to anonymous if `REQUIRE_API_KEY != "true"`. Allows dashboard-session GET on `/api/v1/models` (used by the dashboard model catalog).
- **`clientApiPolicy`** (`policies/clientApi.ts`) — extracts Bearer, validates via `validateApiKey()`. Falls through to anonymous only when the effective `REQUIRE_API_KEY` feature flag is disabled. The effective flag is resolved through `isRequireApiKeyEnabled()` (`DB feature flag override > process.env.REQUIRE_API_KEY > default`) so Dashboard Feature Flags and environment variables govern `/api/v1/*` and aliases consistently; resolver failures fail closed. Allows dashboard-session requests on client API routes (including `/api/v1/models`, used by the dashboard model catalog).
- **`managementPolicy`** (`policies/management.ts`) — accepts dashboard session, internal model-sync requests (matched against `/api/providers/[name]/(sync-models|models)`), or skips entirely if `isAuthRequired()` returns false. Returns 403 (`AUTH_001`) when a Bearer token is present but invalid, 401 otherwise. Also enforces the route-guard tiers (LOCAL_ONLY / ALWAYS_PROTECTED) before any auth branch — see [Route Guard Tiers](../security/ROUTE_GUARD_TIERS.md). LOCAL_ONLY paths in `LOCAL_ONLY_MANAGE_SCOPE_BYPASS_PREFIXES` (today: `/api/mcp/`) may be accessed from non-loopback when the Bearer key carries the `manage` scope; all other LOCAL_ONLY paths remain strict-loopback regardless of scope.
A successful policy returns `AuthSubject` with `kind ∈ { client_api_key, dashboard_session, management_key, anonymous }`. Downstream handlers can read it via `assertAuth(request, "CLIENT_API")` in `src/server/authz/assertAuth.ts` instead of re-running auth logic.
@@ -170,6 +170,8 @@ Preset bundles (`MCP_SCOPE_PRESETS`): `readonly`, `full`, `monitor`, `agent`. Us
- No password configured **and** no `INITIAL_PASSWORD` env var → bootstrap mode allows the onboarding wizard and loopback requests, but exposed network requests still need credentials.
- Any DB error → fails closed (secure-by-default).
Client API key enforcement uses `isRequireApiKeyEnabled()` in `src/shared/utils/featureFlags.ts`, not a direct `process.env.REQUIRE_API_KEY` read. This matters for deployed instances: toggling `REQUIRE_API_KEY` in Dashboard → Feature Flags stores a DB override and immediately affects `/v1/*`, `/models`, `/responses`, `/chat/completions`, `/codex/*`, and other client-API auth checks that share this helper. If the feature flag store cannot be read, client API auth fails closed and requires a key.
## Breaking Change — v3.8.0
The `/api/v1/agents/tasks/*` and `/api/resilience/model-cooldowns` endpoints **now require management auth** (commit `588a0333`). Clients previously sending a normal API key without the `manage` scope receive `403`. Migration: either issue the key the `manage` scope in the API Manager dashboard, or use a logged-in dashboard session.

274
Auto-Combo.md

@@ -3,6 +3,8 @@
# OmniRoute Auto-Combo Engine
> **For Users**: Looking for a quick start? See the [Auto-Combo User Guide](../getting-started/AUTO-COMBO-GUIDE.md) for simple explanations and examples.
> Self-managing model chains with adaptive scoring + zero-config auto-routing
## Zero-Config Auto-Routing (`auto/` prefix)
@@ -104,11 +106,11 @@ Four pre-defined weight profiles in `open-sse/services/autoCombo/modePacks.ts`.
| Factor | ship-fast | cost-saver | quality-first | offline-friendly |
| :----------- | :-------- | :--------- | :------------ | :--------------- |
| quota | 0.15 | 0.15 | 0.10 | **0.40** |
| health | 0.30 | 0.20 | 0.20 | 0.30 |
| costInv | 0.05 | **0.40** | 0.05 | 0.10 |
| latencyInv | **0.35** | 0.05 | 0.05 | 0.05 |
| taskFit | 0.10 | 0.10 | **0.40** | 0.00 |
| quota | 0.14 | 0.14 | 0.10 | **0.37** |
| health | 0.28 | 0.19 | 0.18 | 0.28 |
| costInv | 0.05 | **0.37** | 0.05 | 0.10 |
| latencyInv | **0.32** | 0.05 | 0.05 | 0.05 |
| taskFit | 0.10 | 0.10 | **0.37** | 0.00 |
| stability | 0.00 | 0.05 | 0.15 | 0.10 |
| tierPriority | 0.05 | 0.05 | 0.05 | 0.05 |
@@ -116,14 +118,14 @@ Notes:
- `tierAffinity` and `specificityMatch` are not set in mode packs — `calculateScore()` treats them as `?? 0` when absent.
- Each pack's emphasis at a glance:
- **ship-fast** → latencyInv 0.35 + health 0.30 (low-latency, healthy connections)
- **cost-saver** → costInv 0.40 (cheapest tokens win)
- **quality-first** → taskFit 0.40 + stability 0.15 (best model for the task, consistent)
- **offline-friendly** → quota 0.40 + health 0.30 (max headroom regardless of speed/cost)
- **ship-fast** → latencyInv 0.32 + health 0.28 (low-latency, healthy connections)
- **cost-saver** → costInv 0.37 (cheapest tokens win)
- **quality-first** → taskFit 0.37 + stability 0.15 (best model for the task, consistent)
- **offline-friendly** → quota 0.37 + health 0.28 (max headroom regardless of speed/cost)
## All Routing Strategies
OmniRoute's combo engine supports **15 routing strategies** (declared in `src/shared/constants/routingStrategies.ts` → `ROUTING_STRATEGY_VALUES`). The Auto Combo engine itself is exposed under the `auto` strategy; the others are available for persisted combos.
OmniRoute's combo engine supports **14 routing strategies** (declared in `src/shared/constants/routingStrategies.ts` → `ROUTING_STRATEGY_VALUES`). The Auto Combo engine itself is exposed under the `auto` strategy; the others are available for persisted combos.
| Strategy | Description |
| :------------------ | :----------------------------------------------------------------- |
@@ -177,6 +179,8 @@ There is **no dedicated `POST /api/combos/auto` endpoint** — Auto-Combo is con
2. **Persisted combo with `strategy: "auto"`:** Create a regular combo via `POST /api/combos` and set `strategy: "auto"` plus `config.auto.weights` / `config.auto.candidatePool`. The same scoring engine is used; the combo is stored in `combos` and reusable by ID.
For discovery, `GET /api/combos/auto` lists every variant with its resolved candidate pool plus `context_length` / `max_output_tokens` — the MAX across the candidate pool's windows. Clients (e.g. the opencode plugin) must advertise these values instead of `0`: a zero context disables opencode's auto-compaction entirely, letting sessions grow until the gateway's history purge destroys context. MAX is safe to advertise because the auto-combo context pre-filter routes oversized requests to large-window candidates.
```bash
# Zero-config usage (no combo creation)
curl -X POST http://localhost:20128/v1/chat/completions \
@@ -190,6 +194,254 @@ curl -X POST http://localhost:20128/api/combos \
-d '{"id":"my-auto","name":"Auto Coder","strategy":"auto","config":{"auto":{"candidatePool":["anthropic","google","openai"],"weights":{"quota":0.15,"health":0.3,"costInv":0.05,"latencyInv":0.35,"taskFit":0.1,"stability":0,"tierPriority":0.05}}}}'
```
### Auto router strategies
Persisted `strategy: "auto"` combos can set `config.routerStrategy` (or legacy
`config.auto.routerStrategy`) to one of:
- `rules` — default weighted scoring
- `cost` / `eco` — cheapest healthy provider
- `latency` / `fast` — lowest p95 latency with reliability penalty
- `sla-aware` / `sla` — prefer candidates that satisfy p95 latency, error-rate, and optional
cost SLOs
- `lkgp` — last known good provider first
### Router strategies in detail
The auto-combo engine exposes 5 pluggable **RouterStrategy** implementations that
you can swap via `config.routerStrategy` (or the legacy `config.auto.routerStrategy`).
Each strategy picks one provider from the candidate pool, given a `RoutingContext`
(task type, tool/vision hints, token estimate, optional SLA policy, optional
last-known-good provider).
#### 1. `rules` (default) — 6-factor weighted scoring
Wraps the existing scoring engine. Filters out `OPEN` circuit-breaker
candidates, then runs `scorePool()` with the current task type and `getTaskFitness()`,
picking the top-scoring provider.
```ts
class RulesStrategyImpl implements RouterStrategy {
readonly name = "rules";
readonly description =
"6-factor weighted scoring: quota, health, cost, latency, taskFit, stability";
select(pool, context) {
const eligible = pool.filter((c) => c.circuitBreakerState !== "OPEN");
const ranked = scorePool(eligible.length > 0 ? eligible : pool, context.taskType, undefined, getTaskFitness);
return { provider: ranked[0].provider, /* ... */ };
}
}
```
**When to use**: Default. Use when you want a balanced trade-off across all signals.
**Alias**: `rules` (no alias)
---
#### 2. `cost` / `eco` — cheapest healthy provider
Sorts the candidate pool by `costPer1MTokens` (ascending) and picks the cheapest.
Filters out `OPEN` candidates first.
```ts
class CostStrategyImpl implements RouterStrategy {
readonly name = "cost";
readonly description = "Always selects cheapest available provider";
select(pool, context) {
const healthy = pool.filter((c) => c.circuitBreakerState !== "OPEN");
const sorted = [...healthy].sort((a, b) => a.costPer1MTokens - b.costPer1MTokens);
return { provider: sorted[0].provider, /* ... */ };
}
}
```
**When to use**: Cost-sensitive workloads, batch processing, or background jobs.
**Aliases**: `cost`, `eco`
---
#### 3. `latency` / `fast` — lowest p95 latency with reliability penalty
Sorts by `p95LatencyMs + (errorRate * 1000)`. The error-rate penalty ensures
unreliable providers are ranked lower even if their nominal latency is low.
```ts
class LatencyStrategyImpl implements RouterStrategy {
readonly name = "latency";
readonly description = "Prioritizes lowest p95 latency with reliability weighting";
select(pool, context) {
const healthy = pool.filter((c) => c.circuitBreakerState !== "OPEN");
const sorted = [...healthy].sort((a, b) =>
(a.p95LatencyMs + a.errorRate * 1000) - (b.p95LatencyMs + b.errorRate * 1000)
);
return { provider: sorted[0].provider, /* ... */ };
}
}
```
**When to use**: Latency-sensitive workloads like real-time chat, autocomplete, or
interactive coding assistants.
**Aliases**: `latency`, `fast`
---
#### 4. `sla-aware` / `sla` — latency/error/cost SLO compliance
Scores each candidate by how well it satisfies the configured SLO policy:
| Factor | Weight | Formula |
|--------|--------|---------|
| Latency score | 35% | `threshold / max(value, ε)` |
| Error score | 35% | `threshold / max(value, ε)` |
| Health score | 15% | `1.0` (CLOSED) / `0.5` (HALF_OPEN) / `0.0` (OPEN) |
| Cost score | 10% | `threshold / max(value, ε)` or inverse normalized |
| Stability score | 5% | inverse normalized latency stddev |
When `hardConstraints: true`, candidates are sorted primarily by **violation score**
(how far they exceed any SLO), then by composite score. Otherwise it's just
the composite score.
```ts
class SLAStrategyImpl implements RouterStrategy {
readonly name = "sla-aware";
readonly description = "Selects the provider most likely to satisfy latency, error-rate, and cost SLOs";
select(pool, context) {
// ... scores each candidate against policy: { targetP95Ms, maxErrorRate, maxCostPer1MTokens, hardConstraints }
}
}
```
**SLA fields** (set on the combo config):
```json
{
"strategy": "auto",
"config": {
"routerStrategy": "sla-aware",
"slaTargetP95Ms": 1500,
"slaMaxErrorRate": 0.05,
"slaMaxCostPer1MTokens": 5,
"slaHardConstraints": true
}
}
```
**When to use**: Production workloads with strict latency, error-rate, or cost budgets.
**Aliases**: `sla-aware`, `sla`
---
#### 5. `lkgp` — last known good provider first
Tries the **last known good provider** (if set) first, then falls back to the
`rules` strategy. Useful for session stickiness — the same provider handles
follow-up requests in a conversation.
```ts
class LKGPStrategyImpl implements RouterStrategy {
readonly name = "lkgp";
readonly description = "Tries last known good provider first, then falls back to rules";
select(pool, context) {
if (context.lkgpEnabled === false) {
return getStrategy("rules").select(pool, context);
}
if (context.lastKnownGoodProvider) {
const candidates = pool.filter(
(c) => c.provider === context.lastKnownGoodProvider && c.circuitBreakerState !== "OPEN"
);
if (candidates.length > 0) {
return { provider: candidates[0].provider, /* ... */ };
}
}
// Fallback to rules strategy
return getStrategy("rules").select(pool, context);
}
}
```
**When to use**: Multi-turn conversations where you want the same provider to handle
follow-up requests (e.g., for caching, context continuity, or pricing consistency).
**Alias**: `lkgp` (no alias)
---
### Custom router strategies
You can register your own `RouterStrategy` implementation via the public API:
```ts
import { registerStrategy, type RouterStrategy } from "@omniroute/open-sse/services/autoCombo/routerStrategy";
class MyCustomStrategy implements RouterStrategy {
readonly name = "my-custom";
readonly description = "My custom routing strategy";
select(pool, context) {
// Your routing logic here
return {
provider: pool[0].provider,
model: pool[0].model,
strategy: this.name,
reason: "MyCustomStrategy: ...",
candidatesConsidered: pool.length,
finalScore: 1.0,
};
}
}
registerStrategy("my-custom", new MyCustomStrategy());
```
Then use it:
```json
{
"strategy": "auto",
"config": {
"routerStrategy": "my-custom"
}
}
```
---
### Router strategy selection guide
| Use case | Strategy | Reason |
|---------|----------|--------|
| Balanced workload | `rules` | Default — considers all factors |
| Minimize cost | `cost` | Always picks cheapest |
| Minimize latency | `latency` | Picks fastest reliable provider |
| Strict SLOs | `sla-aware` | Filters by p95/error/cost thresholds |
| Multi-turn chat | `lkgp` | Session stickiness |
SLA-aware fields:
```json
{
"strategy": "auto",
"config": {
"routerStrategy": "sla-aware",
"slaTargetP95Ms": 1500,
"slaMaxErrorRate": 0.05,
"slaMaxCostPer1MTokens": 5,
"slaHardConstraints": true
}
}
```
## Task Fitness
30+ models scored across 6 task types (`coding`, `review`, `planning`, `analysis`, `debugging`, `documentation`). Supports wildcard patterns (e.g., `*-coder` → high coding score).
@@ -245,5 +497,5 @@ See `docs/marketing/TIERS.md` for tier definitions and provider classification.
| `open-sse/services/autoCombo/autoPrefix.ts` | `auto/` prefix parser + 6 variants |
| `open-sse/services/autoCombo/virtualFactory.ts` | Builds in-memory `AutoComboConfig` from live connections |
| `open-sse/services/autoCombo/providerRegistryAccessor.ts` | Test hook for mocking provider registry |
| `src/shared/constants/routingStrategies.ts` | `ROUTING_STRATEGY_VALUES` (15 strategies) |
| `src/shared/constants/routingStrategies.ts` | `ROUTING_STRATEGY_VALUES` (14 strategies) |
| `src/sse/handlers/chat.ts` | Integration: auto-prefix short-circuit |

1
CLI-Token-Auth.md

@@ -1,5 +1,6 @@
> 🌍 [View in other languages](Languages)
# CLI Machine-ID Token Authentication
OmniRoute's CLI uses a **machine-derived token** to authenticate to the local server without requiring an explicit API key. This enables zero-config local use while preserving security for remote access.

558
CLI-Tools.md

@@ -1,27 +1,36 @@
> 🌍 [View in other languages](Languages)
# CLI Tools — OmniRoute v3.8.0
# CLI Tools — OmniRoute v3.8.6
Last updated: 2026-05-13
Last updated: 2026-05-28
OmniRoute integrates with two categories of CLI tools:
OmniRoute integrates with three categories of CLI tools spread across three dedicated dashboard pages:
1. **External CLI integrations** — third-party CLIs (Cursor, Cline, Codex, Claude Code, Qwen Code, Windsurf, Hermes, Amp, etc.) that you point at OmniRoute's local OpenAI-compatible endpoint.
2. **Internal OmniRoute CLI** — commands bundled with the `omniroute` binary for server lifecycle, setup, diagnostics, and provider management.
| Page | Route | Concept | Count |
|------|-------|---------|-------|
| **CLI Code's** | `/dashboard/cli-code` | Coding tools you point at OmniRoute (Client → CLI → OmniRoute → Provider) | 19 |
| **CLI Agents** | `/dashboard/cli-agents` | Autonomous agents you point at OmniRoute (same flow, broader scope) | 6 |
| **ACP Agents** | `/dashboard/acp-agents` | CLIs that OmniRoute spawns as backend via stdio/ACP (reverse flow) | see registry |
Legacy routes redirect via 308: `/dashboard/cli-tools` → `/dashboard/cli-code`, `/dashboard/agents` → `/dashboard/acp-agents`.
---
## How It Works
```
Claude / Codex / OpenCode / Cline / KiloCode / Continue / Cursor / Windsurf / Hermes / Amp / Qwen
CLI Code's / CLI Agents (consumption flow):
Claude / Codex / OpenCode / Cline / KiloCode / Continue / Hermes Agent / Goose / ...
│
▼ (all point to OmniRoute)
http://YOUR_SERVER:20128/v1
│
▼ (OmniRoute routes to the right provider)
Anthropic / OpenAI / Gemini / DeepSeek / Groq / Mistral / ...
ACP Agents (reverse spawn flow):
Client request → OmniRoute → spawns CLI via stdio/ACP → response
```
**Benefits:**
@@ -33,70 +42,197 @@ Claude / Codex / OpenCode / Cline / KiloCode / Continue / Cursor / Windsurf / He
---
## 1. External CLI Integrations
## Source of Truth
### Source of Truth
The unified catalog lives in `src/shared/constants/cliTools.ts` as `CLI_TOOLS: Record<string, CliCatalogEntry>`.
The dashboard cards in `/dashboard/cli-tools` are generated from
`src/shared/constants/cliTools.ts`. The `omniroute setup` command can write
config files automatically for the scriptable tools.
Each entry has these fields (defined in `src/shared/schemas/cliCatalog.ts`):
### Current Catalog (v3.8.0)
| Field | Type | Description |
|-------|------|-------------|
| `category` | `"code" \| "agent"` | Which page the tool appears on |
| `vendor` | `string` | Tool origin ("Anthropic", "OSS (P. Gauthier)") |
| `acpSpawnable` | `boolean` | Also usable as an ACP Agent (badge shown) |
| `baseUrlSupport` | `"full" \| "partial" \| "none"` | Custom endpoint support level. `"none"` = MITM backlog |
| `configType` | `"env" \| "custom" \| "guide" \| "custom-builder" \| "mitm"` | Configuration mechanism |
| `id`, `name`, `color`, `description`, `docsUrl` | standard | Core display fields |
| Tool | ID | Type / Config | Install / Access | Auth |
| ------------------ | ------------- | ------------------- | ------------------------------------ | ----------------------------------- |
| **Claude Code** | `claude` | env / settings.json | `npm i -g @anthropic-ai/claude-code` | API key (Anthropic gateway) |
| **OpenAI Codex** | `codex` | custom (toml) | `npm i -g @openai/codex` | API key (OpenAI) |
| **Factory Droid** | `droid` | custom | bundled / CLI | API key |
| **Open Claw** | `openclaw` | custom | bundled / CLI | API key |
| **Cursor** | `cursor` | guide (Cloud) | Cursor desktop app | API key (Cloud Endpoint) |
| **Windsurf** | `windsurf` | guide | Windsurf desktop IDE | API key (BYOK) |
| **Cline** | `cline` | custom / VS Code | `npm i -g cline` + VS Code ext | API key |
| **Kilo Code** | `kilo` | custom / VS Code | `npm i -g kilocode` + VS Code ext | API key |
| **Continue** | `continue` | guide (config.yaml) | VS Code extension | API key |
| **Antigravity** | `antigravity` | MITM | OmniRoute built-in | API key (MITM proxy) |
| **GitHub Copilot** | `copilot` | custom / VS Code | VS Code extension | API key (CLI fingerprint: `github`) |
| **OpenCode** | `opencode` | guide (json) | `npm i -g opencode-ai` | API key (OpenAI-compatible) |
| **Hermes** | `hermes` | guide (json) | install per docs | API key (OpenAI-compatible) |
| **Amp CLI** | `amp` | guide (env) | install per Sourcegraph docs | API key (OpenAI-compatible) |
| **Kiro AI** | `kiro` | MITM | Amazon Kiro IDE / CLI | API key (MITM proxy) |
| **Qwen Code** | `qwen` | guide (json/env) | `npm i -g @qwen-code/qwen-code` | API key (OpenAI-compatible) |
| **Custom CLI** | `custom` | custom-builder | any OpenAI-compatible client | API key |
> Notes:
>
> - "Web wrappers" like ChatGPT/Claude/Grok/Perplexity browser sessions are not
> listed here. OmniRoute can proxy them through the `chatgpt-web`,
> `claude-web`, `grok-web`, `perplexity-web`, `blackbox-web`,
> `muse-spark-web` provider connections, but those are **provider connections**
> (configured under `/dashboard/providers`), not CLI tools. They do not surface
> as cards under `/dashboard/cli-tools`.
> - Tools marked **MITM** (Antigravity, Kiro) intercept the desktop app traffic
> locally and require enabling the corresponding mitm endpoint in
> `/dashboard/settings`.
### CLI fingerprint sync (Agents + Settings)
`/dashboard/agents` and `Settings > CLI Fingerprint` use
`src/shared/constants/cliCompatProviders.ts`. This keeps provider IDs aligned
with the CLI cards and legacy IDs.
| CLI ID | Fingerprint Provider ID |
| --------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- |
| `kilo` | `kilocode` |
| `copilot` | `github` |
| `claude` / `codex` / `antigravity` / `kiro` / `cursor` / `windsurf` / `cline` / `opencode` / `hermes` / `amp` / `qwen` / `droid` / `openclaw` | same ID |
Legacy IDs still accepted for compatibility: `copilot`, `kimi-coding`, `qwen`.
Entries with `baseUrlSupport: "none"` are **not shown** in the dashboard pages — they are registered in the MITM backlog for plan 11 (see `_tasks/features-v3.8.6/refactorpages/_orchestration/_plan11-mitm-backlog.md`).
---
## 1. CLI Code's Catalog (19 tools)
Tools that support custom base URL and appear in `/dashboard/cli-code`:
| id | name | vendor | baseUrlSupport | configType | acpSpawnable |
|----|------|--------|---------------|-----------|-------------|
| claude | Claude Code | Anthropic | full | env | true |
| codex | OpenAI Codex CLI | OpenAI | full | custom | true |
| cline | Cline | OSS (ex-Claude Dev) | full | custom | true |
| kilo | Kilo Code | Kilo-Org | full | custom | false |
| roo | Roo Code | Roo (OSS) | full | guide | false |
| continue | Continue | continue.dev | full | guide | false |
| qwen | Qwen Code | Alibaba | full | guide | true |
| aider | Aider | OSS (P. Gauthier) | full | guide | true |
| forge | ForgeCode | Antinomy HQ | full | custom | true |
| jcode | jcode | 1jehuang (OSS) | full | custom | false |
| deepseek-tui | DeepSeek TUI | Hunter Bown (OSS) | full | custom | false |
| opencode | OpenCode | Anomaly (ex-SST) | full | guide | true |
| droid | Factory Droid | Factory AI | partial | guide | false |
| copilot | GitHub Copilot CLI | GitHub/MS | full | custom | false |
| gemini-cli | Gemini CLI | Google | partial | guide | true |
| cursor-cli | Cursor CLI | Anysphere | partial | guide | true |
| smelt | Smelt | leonardcser (OSS) | full | custom | false |
| pi | Pi (pi-coding-agent) | M. Zechner (OSS) | full | custom | false |
| custom | Custom CLI | — | full | custom-builder | false |
Tools with `baseUrlSupport: "partial"` show a badge "⚠ Base URL parcial" in the dashboard card.
---
## 2. CLI Agents Catalog (6 tools)
Autonomous agents that appear in `/dashboard/cli-agents`:
| id | name | vendor | baseUrlSupport | acpSpawnable |
|----|------|--------|---------------|-------------|
| hermes-agent | Hermes Agent | Nous Research | full | false |
| openclaw | OpenClaw | OSS (P. Steinberger) | full | true |
| goose | Goose | Block / Linux Foundation | full | true |
| interpreter | Open Interpreter | OSS | full | true |
| warp | Warp AI | Warp Inc. | partial | true |
| agent-deck | Agent Deck | asheshgoplani (OSS) | full | false |
---
## 3. ACP Agents (/dashboard/acp-agents)
This page (renamed from `/dashboard/agents`) shows CLIs that OmniRoute can **spawn** as backend execution engines via stdio/ACP protocol. The catalog is maintained separately in `src/lib/acp/registry.ts` and is **not** the same as `CLI_TOOLS`.
Current ACP-spawnable CLIs (from `acpSpawnable: true` in `CLI_TOOLS` + ACP registry): codex, claude, goose, gemini-cli, openclaw, aider, opencode, cline, qwen-code, forge, interpreter, cursor-cli, warp.
---
## 4. MITM Backlog (not shown in dashboard)
The following CLIs do not support custom base URL natively and are **not listed** in CLI Code's or CLI Agents pages. They are candidates for MITM interception in plan 11:
| CLI | Reason |
|-----|--------|
| windsurf | BYOK limited to select Claude models + corporate URL/token |
| amp | Closed ecosystem (Sourcegraph) |
| amazon-q / kiro-cli | AWS SSO auth, no custom URL |
| cowork | Anthropic Desktop, no configurable endpoint |
See `_tasks/features-v3.8.6/refactorpages/_orchestration/_plan11-mitm-backlog.md` for the full cross-reference.
---
## 5. Batch Detection API
All tool detection is aggregated via a single endpoint:
**`GET /api/cli-tools/all-statuses`**
- Auth: `requireCliToolsAuth(request)` (same as other `/api/cli-tools/` routes)
- Returns: `Record<toolId, ToolBatchStatus>` (type: `src/shared/types/cliBatchStatus.ts`)
- Strategy: `Promise.all` over all tools, 5s timeout per tool
- Cache: in-memory LRU indexed by config file `mtime`. Cache invalidated when mtime changes. Reset on server restart.
Response shape per tool:
```ts
interface ToolBatchStatus {
detection: {
installed: boolean;
runnable: boolean;
version?: string;
command?: string;
commandPath?: string;
reason?: string;
};
config: {
status: "configured" | "not_configured" | "not_installed" | "unknown" | "other";
endpoint?: string | null;
lastConfiguredAt?: string | null;
};
error?: string; // sanitized, no stack traces
}
```
---
## 6. Settings Handlers for New Tools
New tools with `configType: "custom"` have dedicated settings API routes:
| Route | Tool |
|-------|------|
| `POST /api/cli-tools/forge-settings` | ForgeCode (.forge.toml) |
| `POST /api/cli-tools/jcode-settings` | jcode (--base-url flag) |
| `POST /api/cli-tools/deepseek-tui-settings` | DeepSeek TUI (OPENAI_BASE_URL) |
| `POST /api/cli-tools/smelt-settings` | Smelt |
| `POST /api/cli-tools/pi-settings` | Pi coding agent |
All routes use `sanitizeErrorMessage()` for error responses (Hard Rule #12).
---
## 7. Dashboard Pages Architecture
### CLI Code's (`/dashboard/cli-code`)
- `src/app/(dashboard)/dashboard/cli-code/page.tsx` — server component
- `src/app/(dashboard)/dashboard/cli-code/CliCodePageClient.tsx` — client grid
- `src/app/(dashboard)/dashboard/cli-code/[id]/page.tsx` — tool detail page
- `src/app/(dashboard)/dashboard/cli-code/components/` — 12 specialized tool cards + `ToolDetailClient.tsx`
### CLI Agents (`/dashboard/cli-agents`)
- `src/app/(dashboard)/dashboard/cli-agents/page.tsx` — server component
- `src/app/(dashboard)/dashboard/cli-agents/CliAgentsPageClient.tsx` — client grid
- `src/app/(dashboard)/dashboard/cli-agents/[id]/page.tsx` — reuses `ToolDetailClient`
### ACP Agents (`/dashboard/acp-agents`)
- `src/app/(dashboard)/dashboard/acp-agents/page.tsx` — server component (moved from `agents/`)
### Shared UI Components (`src/shared/components/cli/`)
| File | Purpose |
|------|---------|
| `CliToolCard.tsx` | Smart status card (detection + config + endpoint) |
| `CliConceptCard.tsx` | Per-page concept explanation card |
| `CliComparisonCard.tsx` | Three-column comparison across CLI types |
| `BaseUrlSelect.tsx` | Endpoint dropdown (Local/Cloud/Custom) |
| `ApiKeySelect.tsx` | API key selector |
| `ManualConfigModal.tsx` | Copiable config snippet modal |
### Shared Hook (`src/shared/hooks/cli/`)
| File | Purpose |
|------|---------|
| `useToolBatchStatuses.ts` | Fetches `/api/cli-tools/all-statuses`, manages loading/refresh state |
---
## 8. i18n
New namespaces added in plan 14 F9:
| Namespace | Purpose |
|-----------|---------|
| `cliCommon` | Shared strings (card labels, concept/comparison texts, detail page labels) |
| `cliCode` | CLI Code's page strings |
| `cliAgents` | CLI Agents page strings |
| `acpAgents` | ACP Agents page strings |
Full PT-BR and EN translations are provided. 39 other locales fall back to EN automatically via namespace-level merge in `src/i18n/request.ts`.
---
## 9. Quick Start
### Step 1 — Get an OmniRoute API Key
1. Open the OmniRoute dashboard → **API Manager** (`/dashboard/api-manager`)
2. Click **Create API Key**
3. Give it a name (e.g. `cli-tools`) and select all permissions
4. Copy the key — you'll need it for every CLI below
1. Open `/dashboard/api-manager` → **Create API Key**
2. Give it a name (e.g. `cli-tools`) and select all permissions
3. Copy the key — you'll need it for every CLI below
> Your key looks like: `sk-xxxxxxxxxxxxxxxx-xxxxxxxxx`
@@ -125,29 +261,32 @@ npm install -g kilocode
# Qwen Code (Alibaba)
npm install -g @qwen-code/qwen-code
# Kiro CLI (Amazon — requires curl + unzip)
apt-get install -y unzip # on Debian/Ubuntu
curl -fsSL https://cli.kiro.dev/install | bash
export PATH="$HOME/.local/bin:$PATH" # add to ~/.bashrc
```
# Aider
pip install aider-chat
**Verify:**
# Smelt
cargo install smelt # Rust-based
```bash
claude --version # 2.x.x
codex --version # 0.x.x
opencode --version # x.x.x
cline --version # 2.x.x
kilocode --version # x.x.x (or: kilo --version)
qwen --version # x.x.x
kiro-cli --version # 1.x.x
# Pi coding agent
# see https://github.com/zechnerj/pi-coding-agent for install
# jcode
# see https://github.com/1jehuang/jcode for install
```
---
### Step 3 — Set Global Environment Variables
### Step 3 — Configure via Dashboard
Add to `~/.bashrc` (or `~/.zshrc`), then run `source ~/.bashrc`:
1. Go to `http://localhost:20128/dashboard/cli-code`
2. Find your tool in the grid
3. Click the card to open the tool detail page
4. Select your API key and base URL
5. Click **Apply Config** or copy the manual config snippet
---
### Step 4 — Set Global Environment Variables
```bash
# OmniRoute Universal Endpoint
@@ -160,7 +299,7 @@ export GEMINI_API_KEY="sk-your-omniroute-key"
```
> For a **remote server** replace `localhost:20128` with the server IP or domain,
> e.g. `http://192.168.0.15:20128`.
> e.g. `http://<your-server-ip>:20128`.
---
@@ -292,6 +431,47 @@ Restart VS Code after editing.
---
#### VS Code Insiders (`chatLanguageModels.json`)
Use this when VS Code Insiders is configured for custom endpoint models and you want OmniRoute to work without a custom header field.
**Recommended location:**
- Linux: `~/.config/Code - Insiders/User/chatLanguageModels.json`
- Windows: `%APPDATA%/Code - Insiders/User/chatLanguageModels.json`
**Example using the tokenized OmniRoute alias:**
```json
[
{
"vendor": "customendpoint",
"id": "auto",
"name": "OmniRoute Auto",
"family": "gpt-4",
"version": "1.0.0",
"url": "http://localhost:20128/api/v1/vscode/sk-your-omniroute-key/chat/completions",
"modelsUrl": "http://localhost:20128/api/v1/vscode/sk-your-omniroute-key/models",
"requestFormat": "openai-chat-completions",
"contextWindow": 256000,
"maxOutputTokens": 32768,
"auth": {
"type": "none"
}
}
]
```
**Notes:**
- Replace `sk-your-omniroute-key` with an API key created in OmniRoute.
- The `url` field should point to `/api/v1/vscode/{token}/chat/completions`.
- The `modelsUrl` field should point to `/api/v1/vscode/{token}/models`.
- Prefer the normal `/v1` + Bearer header flow when the client supports custom headers.
- URL-embedded tokens are a compatibility fallback and may appear in editor logs or proxy history.
---
#### Kiro CLI (Amazon)
```bash
@@ -355,117 +535,21 @@ qwen
> For a **remote server** replace `localhost:20128` with the server IP or domain.
**Test:** `qwen "say hello"`
---
#### Cursor (Desktop App)
## 10. Internal OmniRoute CLI
> **Note:** Cursor routes requests through its cloud. For OmniRoute integration,
> enable **Cloud Endpoint** in OmniRoute Settings and use your public domain URL.
Via GUI: **Settings → Models → OpenAI API Key**
- Base URL: `https://your-domain.com/v1`
- API Key: your OmniRoute key
---
#### Windsurf (Desktop IDE)
> Official Windsurf docs currently describe BYOK for select Claude models plus
> enterprise URL/token settings, not a generic custom OpenAI-compatible provider.
> Test BYOK behavior in your environment before relying on this integration.
1. Open AI Settings inside Windsurf.
2. Select **Add custom provider** (OpenAI-compatible).
3. Base URL: `http://localhost:20128/v1`
4. API Key: your OmniRoute key
5. Pick a model from the OmniRoute catalog.
---
#### Hermes
```json
// Hermes config file
{
"provider": {
"type": "openai",
"baseURL": "http://localhost:20128/v1",
"apiKey": "sk-your-omniroute-key",
"model": "claude-sonnet-4-6"
}
}
```
---
#### Amp CLI (Sourcegraph)
```bash
export OPENAI_API_KEY="sk-your-omniroute-key"
export OPENAI_BASE_URL="http://localhost:20128/v1"
amp --model "claude-sonnet-4-6"
# Suggested shorthand aliases you can map locally:
# g25p -> gemini/gemini-2.5-pro
# g25f -> gemini/gemini-2.5-flash
# cs45 -> cc/claude-sonnet-4-5-20250929
# g54 -> gemini/gemini-3.1-pro-high
```
---
### Dashboard Auto-Configuration
The OmniRoute dashboard automates configuration for most tools:
1. Go to `http://localhost:20128/dashboard/cli-tools`
2. Expand any tool card
3. Select your API key from the dropdown
4. Click **Apply Config** (if the tool is detected as installed)
5. Or copy the generated config snippet manually
---
### Built-in Agents: Droid & Open Claw
**Droid** and **Open Claw** are AI agents built directly into OmniRoute — no
installation needed. They run as internal routes and use OmniRoute's model
routing automatically.
- Access: `http://localhost:20128/dashboard/agents`
- Configure: same combos and providers as all other tools
- No API key or CLI install required
---
## 2. Internal OmniRoute CLI
The `omniroute` binary (installed via `npm install -g omniroute` or bundled
with the desktop app) provides commands beyond running the server. The full
matrix is implemented in:
- `bin/omniroute.mjs` — entry point, env loading, special-case dispatch (`--mcp`)
- `bin/cli/program.mjs` — Commander program builder
- `bin/cli/commands/<cmd>.mjs` — one file per command/group, registered in `registry.mjs`
- `bin/cli/output.mjs` — output formatters (json/jsonl/table/csv)
- `bin/cli/runtime.mjs` — withRuntime helper (server-first/db-fallback)
- `bin/cli/i18n.mjs` — t() helper with locales
### Server Lifecycle
The `omniroute` binary provides commands for server lifecycle, setup, diagnostics, and provider management. Entry point: `bin/omniroute.mjs`.
```bash
omniroute # Start server (default port 20128)
omniroute --port 3000 # Override port
omniroute --no-open # Don't auto-open browser
omniroute --mcp # Start as MCP server (stdio transport)
omniroute serve # Same as `omniroute`
omniroute stop # Stop the running server
omniroute restart # Restart the server
omniroute dashboard # Open dashboard in default browser
omniroute open # Alias for `dashboard`
omniroute setup # Interactive setup wizard
omniroute doctor # Check config, DB, ports, runtime
omniroute providers list # Configured provider connections
omniroute providers test-all # Test every active connection
omniroute reset-password # Reset the admin password
omniroute logs # Stream request logs
omniroute health # Detailed health (breakers, cache, memory)
omniroute --version # Print version
omniroute --help # Show all commands
```
@@ -595,93 +679,29 @@ omniroute completion # Generate shell completion
| `/v1/audio/speech` | Text-to-speech | ElevenLabs, OpenAI TTS |
| `/v1/audio/transcriptions` | Speech-to-text | Deepgram, AssemblyAI |
Ready-to-paste examples with a tokenized OmniRoute URL:
```txt
Token example: sk-a3ab3c080beaee3a-69f4a4-070d71af
Standard OpenAI base: http://localhost:20128/v1
VS Code models: http://localhost:20128/api/v1/vscode/sk-a3ab3c080beaee3a-69f4a4-070d71af/models
VS Code chat: http://localhost:20128/api/v1/vscode/sk-a3ab3c080beaee3a-69f4a4-070d71af/chat/completions
VS Code responses: http://localhost:20128/api/v1/vscode/sk-a3ab3c080beaee3a-69f4a4-070d71af/responses
Ollama tags: http://localhost:20128/api/v1/vscode/sk-a3ab3c080beaee3a-69f4a4-070d71af/api/tags
Ollama chat: http://localhost:20128/api/v1/vscode/sk-a3ab3c080beaee3a-69f4a4-070d71af/api/chat
```
---
## Troubleshooting
| Error | Cause | Fix |
| ------------------------------------------------- | --------------------------- | --------------------------------------------------------------------------- |
| `Connection refused` | OmniRoute not running | `omniroute serve` or `pm2 start omniroute` |
| `401 Unauthorized` | Wrong API key | Check in `/dashboard/api-manager` |
| `No combo configured` | No active routing combo | Set up in `/dashboard/combos` |
| `invalid model` | Model not in catalog | Use `auto` or check `/dashboard/providers` |
| CLI shows "not installed" | Binary not in PATH | Check `which <command>` |
| `kiro-cli: not found` | Not in PATH | `export PATH="$HOME/.local/bin:$PATH"` |
| `doctor` reports SQLite incompatible | Wrong native binary | `cd app && npm rebuild better-sqlite3` |
| `doctor` reports `STORAGE_ENCRYPTION_KEY` missing | Encrypted creds without key | Set `STORAGE_ENCRYPTION_KEY` or `omniroute reset-encrypted-columns --force` |
---
## Quick Setup Script (One Command)
```bash
cat > my-setup.sh <<'EOF'
#!/usr/bin/env bash
set -euo pipefail
# === Edit these ===
OMNIROUTE_URL="http://localhost:20128/v1"
OMNIROUTE_ANTHROPIC_URL="http://localhost:20128"
OMNIROUTE_KEY="sk-your-omniroute-key"
# ==================
# 1. Install the external CLIs
npm install -g \
@anthropic-ai/claude-code \
@openai/codex \
opencode-ai \
cline \
kilocode \
@qwen-code/qwen-code
# 2. Optional: Kiro CLI (needs unzip)
if ! command -v unzip >/dev/null 2>&1; then
sudo apt-get install -y unzip
fi
curl -fsSL https://cli.kiro.dev/install | bash
# 3. Write the per-tool config files
mkdir -p ~/.claude ~/.codex ~/.config/opencode ~/.continue ~/.qwen
cat > ~/.claude/settings.json <<JSON
{
"env": {
"ANTHROPIC_BASE_URL": "${OMNIROUTE_ANTHROPIC_URL}",
"ANTHROPIC_AUTH_TOKEN": "${OMNIROUTE_KEY}"
}
}
JSON
cat > ~/.codex/config.yaml <<YAML
model: auto
apiKey: ${OMNIROUTE_KEY}
apiBaseUrl: ${OMNIROUTE_URL}
YAML
cat > ~/.qwen/.env <<ENV
OPENAI_API_KEY="${OMNIROUTE_KEY}"
OPENAI_BASE_URL="${OMNIROUTE_URL}"
OPENAI_MODEL="auto"
ENV
# 4. Append global env vars (idempotent guard)
if ! grep -q "OmniRoute Universal Endpoint" ~/.bashrc 2>/dev/null; then
cat >> ~/.bashrc <<ENV
# OmniRoute Universal Endpoint
export OPENAI_BASE_URL="${OMNIROUTE_URL}"
export OPENAI_API_KEY="${OMNIROUTE_KEY}"
export ANTHROPIC_BASE_URL="${OMNIROUTE_ANTHROPIC_URL}"
export ANTHROPIC_AUTH_TOKEN="${OMNIROUTE_KEY}"
ENV
fi
# 5. Validate via the internal CLI
omniroute doctor || true
omniroute providers list || true
echo "All CLIs installed and configured for OmniRoute"
EOF
chmod +x my-setup.sh
./my-setup.sh
```
| Error | Cause | Fix |
|-------|-------|-----|
| `Connection refused` | OmniRoute not running | `omniroute serve` |
| `401 Unauthorized` | Wrong API key | Check in `/dashboard/api-manager` |
| `No combo configured` | No active routing combo | Set up in `/dashboard/combos` |
| CLI shows "not installed" | Binary not in PATH | Check `which <command>` |
| Dashboard shows "not detected" after install | Cache stale | Click "⟳ Refresh detection" in dashboard |
| Old link `/dashboard/cli-tools` | Pre-v3.8.6 bookmark | Auto-redirected to `/dashboard/cli-code` (308) |
| Old link `/dashboard/agents` | Pre-v3.8.6 bookmark | Auto-redirected to `/dashboard/acp-agents` (308) |

123
Codebase-Documentation.md

@@ -174,6 +174,7 @@ src/app/api/
├── token-health/
├── translator/
├── tunnels/
├── services/ Embedded service management (9router, cliproxy) — LOCAL_ONLY
├── upstream-proxy/
├── usage/
├── v1/ OpenAI-compatible public API
@@ -182,6 +183,44 @@ src/app/api/
└── webhooks/
```
#### 3.1.2a `src/app/api/services/` — Embedded Services management
Routes for installing, starting, stopping, and monitoring 9Router and CLIProxyAPI.
All paths are classified **LOCAL_ONLY** (loopback only, hard rule #17) because they
can invoke `npm install` and spawn child processes.
```
src/app/api/services/
├── 9router/
│ ├── _lib.ts getOrInitSupervisor() helper
│ ├── install/route.ts POST — npm install via execFile
│ ├── start/route.ts POST — supervisor.start()
│ ├── stop/route.ts POST — supervisor.stop()
│ ├── restart/route.ts POST — supervisor.restart()
│ ├── update/route.ts POST — npm install newer version
│ ├── rotate-key/route.ts POST — generate new API key + restart
│ ├── status/route.ts GET — live + DB status + version metadata
│ └── auto-start/route.ts POST — toggle auto_start flag
├── cliproxy/
│ ├── _lib.ts getOrInitSupervisor() helper
│ ├── install/route.ts POST — npm install
│ ├── start/route.ts POST — supervisor.start()
│ ├── stop/route.ts POST — supervisor.stop()
│ ├── restart/route.ts POST — supervisor.restart()
│ ├── update/route.ts POST — npm install newer version
│ ├── status/route.ts GET — live + DB status + version metadata
│ └── auto-start/route.ts POST — toggle auto_start flag
└── [name]/
└── logs/route.ts GET — SSE log tail (shared by all services)
```
Corresponding dashboard UI:
`src/app/(dashboard)/dashboard/providers/services/` — two-tab page (CLIProxyAPI + 9Router).
Reverse proxy for 9Router embedded UI:
`src/app/(dashboard)/dashboard/providers/services/[name]/embed/[...path]/route.ts`
Deep-dive: `docs/frameworks/EMBEDDED-SERVICES.md`
#### 3.1.3 `src/app/api/v1/` — OpenAI-compatible public API
```
@@ -230,44 +269,46 @@ the same `open-sse/handlers/` pipeline).
Always import data, sync, OAuth, skill, memory, etc. through these modules. The
table groups the actual directories and notable top-level files.
| Module | Purpose |
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `a2a/` | A2A protocol server: `taskManager.ts`, `streaming.ts`, `taskExecution.ts`, `routingLogger.ts`, `skills/` (5 skills: cost analysis, health report, provider discovery, quota management, smart routing) |
| `acp/` | Agent-Control-Protocol: `index.ts`, `manager.ts`, `registry.ts` |
| `api/` | Internal API helpers: `requireManagementAuth.ts`, `requireCliToolsAuth.ts`, `errorResponse.ts` |
| `auth/` | `managementPassword.ts` (password reset / hashing) |
| `batches/` | OpenAI Batches API service (`service.ts`) |
| `catalog/` | OpenRouter catalog sync (`openrouterCatalog.ts`) |
| `cloudAgent/` | Cloud agent registry: `api.ts`, `baseAgent.ts`, `db.ts`, `index.ts`, `registry.ts`, `types.ts`, `agents/{codex, devin, jules}.ts` |
| `combos/` | Combo resolution helpers |
| `compliance/` | Audit + provider audit: `index.ts`, `providerAudit.ts` |
| `config/` | Runtime config glue |
| `db/` | SQLite domain modules (see §3.2.1) |
| `display/` | UI/display helpers used by API responses |
| `embeddings/` | Embedding service registry |
| `env/` | Env loading + introspection |
| `evals/` | Eval runtime |
| `guardrails/` | `piiMasker.ts`, `promptInjection.ts`, `visionBridge.ts`, `visionBridgeHelpers.ts`, `registry.ts`, `base.ts` |
| `jobs/` | Background jobs (`autoUpdate.ts`, …) |
| `memory/` | Persistent memory: `store.ts`, `cache.ts`, `retrieval.ts`, `summarization.ts`, `extraction.ts`, `injection.ts`, `qdrant.ts`, `settings.ts`, `verify.ts`, `schemas.ts`, `types.ts` |
| `monitoring/` | `observability.ts` |
| `oauth/` | OAuth providers (14): `antigravity`, `claude`, `cline`, `codex`, `cursor`, `gemini`, `github`, `gitlab-duo`, `kilocode`, `kimi-coding`, `kiro`, `qoder`, `qwen`, `windsurf` plus `services/`, `utils/{pkce, server, banner, codexAuthFile, ui}`, `constants/oauth.ts` |
| `plugins/` | Plugin loader (`index.ts`) |
| `promptCache/` | `prefixAnalyzer.ts`, `index.ts` |
| `providerModels/` | Managed model lifecycle: `modelDiscovery.ts`, `managedModelImport.ts`, `managedAvailableModels.ts`, `cursorAgent.ts` |
| `providers/` | Provider helpers: `catalog.ts`, `validation.ts`, `imageValidation.ts`, `claudeExtraUsage.ts`, `codexConnectionDefaults.ts`, `codexFastTier.ts`, `webCookieAuth.ts`, `managedAvailableModels.ts`, `requestDefaults.ts` |
| `resilience/` | `settings.ts` — settings for circuit breaker, cooldown, lockout |
| `runtime/` | Runtime feature detection |
| `search/` | `executeWebSearch.ts` |
| `skills/` | Skill framework: `registry.ts`, `executor.ts`, `interception.ts`, `injection.ts`, `sandbox.ts`, `custom.ts`, `hybrid.ts`, `builtins.ts`, `a2a.ts`, `providerSettings.ts`, `schemas.ts`, `skillssh.ts`, `types.ts`, plus `builtin/browser.ts` |
| `spend/` | `batchWriter.ts` (write-behind buffer) |
| `sync/` | `bundle.ts`, `tokens.ts` (Cloud Sync) |
| `system/` | System-level helpers |
| `translator/` | Top-level translator glue (delegates into `open-sse/translator/`) |
| `usage/` | Usage accounting: `costCalculator.ts`, `tokenAccounting.ts`, `usageHistory.ts`, `aggregateHistory.ts`, `usageStats.ts`, `callLogs.ts`, `callLogArtifacts.ts`, `fetcher.ts`, `providerLimits.ts`, `migrations.ts` |
| `versionManager/` | Auto-update + version manifest |
| `ws/` | WebSocket bridge |
| `zed-oauth/` | Zed editor OAuth flow |
| Module | Purpose |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `a2a/` | A2A protocol server: `taskManager.ts`, `streaming.ts`, `taskExecution.ts`, `routingLogger.ts`, `skills/` (6 skills: cost analysis, health report, provider discovery, quota management, smart routing, list-capabilities) |
| `acp/` | Agent-Control-Protocol: `index.ts`, `manager.ts`, `registry.ts` |
| `api/` | Internal API helpers: `requireManagementAuth.ts`, `requireCliToolsAuth.ts`, `errorResponse.ts` |
| `auth/` | `managementPassword.ts` (password reset / hashing) |
| `batches/` | OpenAI Batches API service (`service.ts`) |
| `catalog/` | OpenRouter catalog sync (`openrouterCatalog.ts`) |
| `cloudAgent/` | Cloud agent registry: `api.ts`, `baseAgent.ts`, `db.ts`, `index.ts`, `registry.ts`, `types.ts`, `agents/{codex, devin, jules}.ts` |
| `combos/` | Combo resolution helpers |
| `compliance/` | Audit + provider audit: `index.ts`, `providerAudit.ts` |
| `config/` | Runtime config glue |
| `db/` | SQLite domain modules (see §3.2.1) |
| `display/` | UI/display helpers used by API responses |
| `embeddings/` | Embedding service registry |
| `env/` | Env loading + introspection |
| `evals/` | Eval runtime |
| `guardrails/` | `piiMasker.ts`, `promptInjection.ts`, `visionBridge.ts`, `visionBridgeHelpers.ts`, `registry.ts`, `base.ts` |
| `jobs/` | Background jobs (`autoUpdate.ts`, …) |
| `memory/` | Persistent memory: `store.ts`, `cache.ts`, `retrieval.ts`, `summarization.ts`, `extraction.ts`, `injection.ts`, `qdrant.ts`, `settings.ts`, `verify.ts`, `schemas.ts`, `types.ts` |
| `monitoring/` | `observability.ts` |
| `oauth/` | OAuth providers (14): `antigravity`, `claude`, `cline`, `codex`, `cursor`, `gemini`, `github`, `gitlab-duo`, `kilocode`, `kimi-coding`, `kiro`, `qoder`, `qwen`, `windsurf` plus `services/`, `utils/{pkce, server, banner, codexAuthFile, ui}`, `constants/oauth.ts` |
| `plugins/` | Plugin loader (`index.ts`) |
| `promptCache/` | `prefixAnalyzer.ts`, `index.ts` |
| `providerModels/` | Managed model lifecycle: `modelDiscovery.ts`, `managedModelImport.ts`, `managedAvailableModels.ts`, `cursorAgent.ts` |
| `providers/` | Provider helpers: `catalog.ts`, `validation.ts`, `imageValidation.ts`, `claudeExtraUsage.ts`, `codexConnectionDefaults.ts`, `codexFastTier.ts`, `webCookieAuth.ts`, `managedAvailableModels.ts`, `requestDefaults.ts` |
| `resilience/` | `settings.ts` — settings for circuit breaker, cooldown, lockout |
| `runtime/` | Runtime feature detection |
| `search/` | `executeWebSearch.ts` |
| `services/` | Embedded services framework: `ServiceSupervisor.ts` (generic child-process supervisor with operation lock, ring buffer, health checker), `bootstrap.ts` (process-level registration and auto-start), `registry.ts` (tool → supervisor map), `apiKey.ts` (AES-256-GCM key store), `modelSync.ts` (periodic model sync), `ringBuffer.ts` (5 MB circular log buffer), `healthCheck.ts` (HTTP health probe), `types.ts`, `embedWsProxy.ts` (WebSocket proxy), `installers/{ninerouter,cliproxy}.ts`. See `docs/frameworks/EMBEDDED-SERVICES.md` |
| `agentSkills/` | Agent Skills catalog + generator: `catalog.ts` (getCatalog/getSkillById/filterCatalog/computeCoverage), `generator.ts` (generateAgentSkills → writes `skills/{id}/SKILL.md`), `openapiParser.ts` (extracts REST endpoints from OpenAPI spec), `cliRegistryParser.ts` (extracts CLI subcommands from bin/cli-registry), `schemas.ts` (Zod: AgentSkillSchema, SkillCoverageSchema, ListQuerySchema, GenerateBodySchema), `types.ts` (AgentSkill, SkillCoverage, SkillMarkdown, GeneratorReport). Consumed by REST routes (`/api/agent-skills/*`), MCP tools (`omniroute_agent_skills_*`), and A2A skill `list-capabilities`. See [AGENT-SKILLS.md](../frameworks/AGENT-SKILLS.md). |
| `skills/` | Skill framework: `registry.ts`, `executor.ts`, `interception.ts`, `injection.ts`, `sandbox.ts`, `custom.ts`, `hybrid.ts`, `builtins.ts`, `a2a.ts`, `providerSettings.ts`, `schemas.ts`, `skillssh.ts`, `types.ts`, plus `builtin/browser.ts` |
| `spend/` | `batchWriter.ts` (write-behind buffer) |
| `sync/` | `bundle.ts`, `tokens.ts` (Cloud Sync) |
| `system/` | System-level helpers |
| `translator/` | Top-level translator glue (delegates into `open-sse/translator/`) |
| `usage/` | Usage accounting: `costCalculator.ts`, `tokenAccounting.ts`, `usageHistory.ts`, `aggregateHistory.ts`, `usageStats.ts`, `callLogs.ts`, `callLogArtifacts.ts`, `fetcher.ts`, `providerLimits.ts`, `migrations.ts` |
| `versionManager/` | Auto-update + version manifest |
| `ws/` | WebSocket bridge |
| `zed-oauth/` | Zed editor OAuth flow |
Top-level files in `src/lib/`:
@@ -408,12 +449,12 @@ open-sse/
├── types.d.ts
├── config/ Provider registries, header profiles, identity, …
├── handlers/ Request handlers (chat, embeddings, audio, image, …)
├── executors/ 38 provider-specific HTTP executors
├── executors/ 45 provider-specific HTTP executors
├── translator/ Format conversion (OpenAI ↔ Claude ↔ Gemini ↔ Cursor ↔ Kiro)
├── transformer/ Responses API ↔ Chat Completions stream transformer
├── services/ 80+ service modules (combos, fallback, quotas, identity, …)
├── utils/ Streaming helpers, TLS client, AWS SigV4, proxy fetch, …
└── mcp-server/ MCP server (3 transports, 13 scopes, 42 tools)
└── mcp-server/ MCP server (3 transports, 30 scopes, 87 tools)
```
### 4.1 `open-sse/handlers/`
@@ -438,7 +479,7 @@ open-sse/
### 4.2 `open-sse/executors/`
38 provider executors, each extending `BaseExecutor` (`base.ts`):
45 provider executors, each extending `BaseExecutor` (`base.ts`):
`antigravity`, `azure-openai`, `blackbox-web`, `chatgpt-web`, `cliproxyapi`,
`cloudflare-ai`, `codex`, `commandCode`, `cursor`, `default`, `devin-cli`,

5
Compliance.md

@@ -115,6 +115,11 @@ from `src/server-init.ts` and `src/instrumentation-node.ts`. Each run logs a
`compliance.cleanup` audit event with the per-table delete counts. Proxy/call
log trimming is batched (`BATCH_SIZE = 5000`) to avoid long write locks.
Manual request-history cleanup is separate from retention. The Request Logs
page calls `POST /api/settings/purge-request-history`, which deletes `call_logs`,
legacy `request_detail_logs`, and local request artifacts under
`${DATA_DIR}/call_logs/`.
Defaults are defined in `src/lib/logEnv.ts`
(`DEFAULT_APP_LOG_RETENTION_DAYS = 7`, `DEFAULT_CALL_LOG_RETENTION_DAYS = 7`).

200
Compression-Guide.md

@@ -268,6 +268,206 @@ RTK mode is inspired by **[RTK - Rust Token Killer](https://github.com/rtk-ai/rt
---
## Advanced Compression Systems
Beyond the 7 standard modes, OmniRoute includes several advanced compression
systems that work automatically based on context.
### Cache-Aware Compression
Some providers (like Anthropic with prompt caching) support **prompt caching**,
which lets them cache parts of the prompt to reduce costs and latency. When
caching is enabled, aggressive compression can actually **hurt** performance
because it changes the cached tokens, invalidating the cache.
The `cachingAware.ts` module solves this by **detecting caching context** and
**adjusting the compression strategy** accordingly.
#### How it works
1. **Detect caching context** — Scans the request body for `cache_control` markers
2. **Identify caching providers** — Checks if the target provider supports caching
3. **Adjust strategy** — Downgrades `aggressive`/`ultra` to `standard` for caching providers
4. **Skip system prompt** — System prompts are usually cached, so don't compress them
5. **Use deterministic transformations** — Only use transformations that produce consistent output
#### Code example
```ts
import { detectCachingContext, getCacheAwareStrategy } from "@omniroute/open-sse/services/compression/cachingAware";
const body = {
model: "anthropic/claude-sonnet-4.5",
messages: [{ role: "user", content: "Hello" }],
cache_control: { type: "ephemeral" }, // ← Cache marker
};
const ctx = detectCachingContext(body, { provider: "anthropic" });
// → { hasCacheControl: true, provider: "anthropic", isCachingProvider: true }
const strategy = getCacheAwareStrategy("aggressive", ctx);
// → { strategy: "standard", skipSystemPrompt: true, deterministicOnly: true }
```
#### When to use
Cache-aware compression is **always on** — no configuration needed. It only kicks in
when:
- The request has `cache_control` markers
- The target provider supports prompt caching (Anthropic, OpenAI, etc.)
### Progressive Aging
Long conversations accumulate many message turns, but older turns become less
relevant. The `progressiveAging.ts` module **degrades messages by turn distance**:
- **Recent turns (0-3)**: Kept verbatim (full detail)
- **Medium turns (4-8)**: Lite compression (whitespace, formatting cleanup)
- **Old turns (9+)**: Caveman compression (filler removal, summarization)
- **Very old turns (20+)**: Heavily summarized or dropped
#### Code example
```ts
import { applyAging } from "@omniroute/open-sse/services/compression/progressiveAging";
const messages = [
{ role: "system", content: "You are a helpful assistant" },
{ role: "user", content: "What is 2+2?" },
{ role: "assistant", content: "4" },
// ... 50 more turns ...
];
const { messages: aged, saved } = applyAging(messages, {
verbatim: 3, // First 3 turns: verbatim
light: 8, // Turns 4-8: lite compression
moderate: 20, // Turns 9-20: caveman compression
// Turns 21+: heavy summarization
});
// saved = number of tokens saved
```
#### When to use
Progressive aging is **always on** for `aggressive` and `ultra` modes. It's
particularly effective for:
- Long-running coding sessions
- Multi-day conversations
- Agentic workflows with many tool calls
### Caveman Output Mode
The `outputMode.ts` module injects **system prompt instructions** to make the
model itself produce compressed, terse output (a "caveman" style).
#### How it works
Instead of compressing the input, this mode adds a system prompt like:
> "Reply in minimal words. Skip pleasantries. Use short sentences."
This works particularly well for:
- Code generation (terser output = fewer tokens)
- Quick Q&A (no need for elaborate explanations)
- Batch processing (maximize throughput)
#### When to use
Caveman output mode is **opt-in** — set it via the combo config:
```json
{
"strategy": "auto",
"config": {
"auto": {
"outputMode": "caveman"
}
}
}
```
### Tool Result Compression
The `toolResultCompressor.ts` module provides **5 specialized compression strategies**
for tool results (function calls, agent outputs, search results, etc.):
1. **Search result compression** — Removes redundant results, keeps top-N
2. **File read compression** — Truncates large files, preserves headers/imports
3. **Code execution compression** — Keeps only essential stdout/stderr
4. **Database query compression** — Limits rows, removes verbose metadata
5. **API response compression** — Strips null fields, condenses arrays
#### When to use
Tool result compression is **always on** when tool calls are present. No
configuration needed.
### Stacked Pipeline
The stacked mode runs **multiple engines in sequence** — usually RTK first
(60-90% savings on tool output), then Caveman (30% additional savings on the
remaining text). This achieves **78-95% total savings**.
#### How it works
```
Input (1000 tokens)
→ RTK (command-aware filter) → 200 tokens
→ Caveman (filler removal) → 140 tokens
→ Output (140 tokens, 86% savings)
```
#### When to use
Use stacked mode for:
- Tool-heavy workflows (agentic coding, research)
- Cost-sensitive batch processing
- When you need maximum token savings
Configure via combo:
```json
{
"strategy": "auto",
"config": {
"auto": {
"modePack": "stacked"
}
}
}
```
---
## Compression Combo Overrides
You can override the global compression mode **per combo** to fine-tune behavior
for different use cases:
```json
{
"id": "coding-combo",
"strategy": "priority",
"config": {
"auto": {
"weights": { "taskFit": 0.5 },
"modePack": "quality-first"
}
},
"compressionOverride": {
"mode": "aggressive",
"stackedPipelines": ["rtk", "caveman"],
"preserveToolDefinitions": true
}
}
```
This is useful for:
- **Coding combos**: Use `aggressive` mode for long sessions
- **Quick Q&A combos**: Use `lite` mode for fast responses
- **Tool-heavy combos**: Use `stacked` mode for max savings
- **Production combos**: Use `cache-aware` mode for caching providers
---
## See Also
- [Environment Config](../reference/ENVIRONMENT.md) — Compression environment variables

6
Compression-Language-Packs.md

@@ -21,12 +21,14 @@ Current shipped packs (verified against `rules/` directory contents):
| ------------------- | -------------- | --------------------------------------------------- |
| English | `rules/en/` | `context`, `dedup`, `filler`, `structural`, `ultra` |
| Spanish | `rules/es/` | `context`, `dedup`, `filler`, `structural`, `ultra` |
| Portuguese (Brazil) | `rules/pt-BR/` | `context`, `filler`, `structural` |
| Portuguese (Brazil) | `rules/pt-BR/` | `context`, `dedup`, `filler`, `structural`, `ultra` |
| German | `rules/de/` | `context`, `filler`, `structural` |
| French | `rules/fr/` | `context`, `filler`, `structural` |
| Japanese | `rules/ja/` | `context`, `filler`, `structural` |
> **Parity note:** `en` and `es` packs have the full 5 categories; `pt-BR`, `de`, `fr`, `ja` ship 3 categories. The missing `dedup` and `ultra` categories silently fall back to the English built-ins. Contributions welcome to add `dedup.json` and `ultra.json` for the smaller packs.
> **Parity note:** `en`, `es`, and `pt-BR` packs have the full 5 categories; `de`, `fr`, `ja` ship 3 categories. The missing `dedup` and `ultra` categories silently fall back to the English built-ins. Contributions welcome to add `dedup.json` and `ultra.json` for the smaller packs.
>
> The `pt-BR` pack is based on **[Troglodita](https://github.com/leninejunior/troglodita)** by Lenine Júnior — a compression system designed from scratch for Brazilian Portuguese grammar (pleonasm reduction, PT-BR filler removal, technical abbreviations for the dev BR community).
>
> The canonical category list and per-category schema live in [`open-sse/services/compression/rules/_schema.json`](../../open-sse/services/compression/rules/_schema.json) (JSON Schema draft 2020-12).

29
Docker-Guide.md

@@ -149,22 +149,29 @@ docker build --target runner-base -t omniroute:base .
docker build --target runner-cli -t omniroute:cli .
```
Defaults exported by `runner-base`: `PORT=20128`, `HOSTNAME=0.0.0.0`, `NODE_OPTIONS=--max-old-space-size=256`, `DATA_DIR=/app/data`, `OMNIROUTE_MIGRATIONS_DIR=/app/migrations`.
Defaults exported by `runner-base`: `PORT=20128`, `HOSTNAME=0.0.0.0`, `NODE_OPTIONS=--max-old-space-size=512`, `DATA_DIR=/app/data`, `OMNIROUTE_MIGRATIONS_DIR=/app/migrations`.
Memory behavior in Docker:
- `NODE_OPTIONS=--max-old-space-size=512` is baked into the image as a fallback.
- The actual server process is started by the standalone launcher, which reads `OMNIROUTE_MEMORY_MB` and appends `--max-old-space-size=<OMNIROUTE_MEMORY_MB>`.
- Node uses the last repeated `--max-old-space-size` value, so setting `OMNIROUTE_MEMORY_MB` controls the effective Docker heap limit.
- If `OMNIROUTE_MEMORY_MB` is unset, the launcher uses `512`.
## Critical Environment Variables
Beyond the defaults documented in [ENVIRONMENT.md](../reference/ENVIRONMENT.md), the following variables matter most when running under Docker:
| Variable | Purpose | Default |
| ----------------------------- | --------------------------------------------------------------------------------------------------- | ------------------------- |
| `OMNIROUTE_WS_BRIDGE_SECRET` | Shared secret for the WebSocket bridge. **Required in production** — set to a strong random string. | unset (must be provided) |
| `REDIS_URL` | Connection string for the rate limiter / cache backend | `redis://redis:6379` |
| `REDIS_PORT` | Host-side port for the bundled Redis container | `6379` |
| `AUTO_UPDATE_HOST_REPO_DIR` | Host path mounted into `cli` profile at `/workspace/omniroute` for self-update workflows | `.` (current directory) |
| `OMNIROUTE_MEMORY_MB` | Node heap ceiling (`NODE_OPTIONS=--max-old-space-size`) baked into the image | `256` (set in Dockerfile) |
| `DASHBOARD_PORT` / `API_PORT` | Override exposed ports for dashboard (20128) and API (20129) | `20128` / `20129` |
| `PROD_DASHBOARD_PORT` | Host-side dashboard port for `docker-compose.prod.yml` | `20130` |
| `CLIPROXYAPI_PORT` | Host-side port for the `cliproxyapi` sidecar | `8317` |
| Variable | Purpose | Default |
| ----------------------------- | --------------------------------------------------------------------------------------------------- | ------------------------ |
| `OMNIROUTE_WS_BRIDGE_SECRET` | Shared secret for the WebSocket bridge. **Required in production** — set to a strong random string. | unset (must be provided) |
| `REDIS_URL` | Connection string for the rate limiter / cache backend | `redis://redis:6379` |
| `REDIS_PORT` | Host-side port for the bundled Redis container | `6379` |
| `AUTO_UPDATE_HOST_REPO_DIR` | Host path mounted into `cli` profile at `/workspace/omniroute` for self-update workflows | `.` (current directory) |
| `OMNIROUTE_MEMORY_MB` | Runtime Node heap ceiling for the Docker standalone server; overrides the image fallback above | `512` |
| `DASHBOARD_PORT` / `API_PORT` | Override exposed ports for dashboard (20128) and API (20129) | `20128` / `20129` |
| `PROD_DASHBOARD_PORT` | Host-side dashboard port for `docker-compose.prod.yml` | `20130` |
| `CLIPROXYAPI_PORT` | Host-side port for the `cliproxyapi` sidecar | `8317` |
## Docker Compose with Caddy (HTTPS Auto-TLS)

575
Environment.md

@@ -49,12 +49,13 @@
These **must** be set before the first run. Without them, the application will either refuse to start or operate with insecure defaults.
| Variable | Required | Default | Source File | Description |
| ---------------------------- | -------------------- | ---------- | -------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `JWT_SECRET` | **Yes** | _(none)_ | `src/lib/auth` | Signs/verifies all dashboard session cookies (JWT). Generate with `openssl rand -base64 48`. |
| `API_KEY_SECRET` | **Yes** | _(none)_ | `src/lib/db/apiKeys.ts` | AES encryption key for API key values at rest in SQLite. Generate with `openssl rand -hex 32`. |
| `INITIAL_PASSWORD` | **Yes** | `CHANGEME` | Bootstrap script | Sets the initial admin dashboard password (matches `.env.example` default — kept obviously insecure to force a change). **Change before first use.** After login, change via Dashboard → Settings → Security. |
| `OMNIROUTE_WS_BRIDGE_SECRET` | **Yes** (production) | _(unset)_ | `src/app/api/internal/codex-responses-ws/route.ts` | Shared secret for the internal Codex Responses WebSocket bridge. Authenticates bridge requests between the Electron/browser WS relay and OmniRoute. ⚠️ **REQUIRED in production — when unset, all WS bridge requests are rejected.** Generate with `openssl rand -base64 32`. |
| Variable | Required | Default | Source File | Description |
| ---------------------------- | -------------------- | ----------------- | -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `JWT_SECRET` | **Yes** | _(none)_ | `src/lib/auth` | Signs/verifies all dashboard session cookies (JWT). Generate with `openssl rand -base64 48`. |
| `API_KEY_SECRET` | **Yes** | _(none)_ | `src/lib/db/apiKeys.ts` | AES encryption key for API key values at rest in SQLite. Generate with `openssl rand -hex 32`. |
| `INITIAL_PASSWORD` | **Yes** | `CHANGEME` | Bootstrap script | Sets the initial admin dashboard password (matches `.env.example` default — kept obviously insecure to force a change). **Change before first use.** After login, change via Dashboard → Settings → Security. |
| `OMNIROUTE_WS_BRIDGE_SECRET` | **Yes** (production) | _(unset)_ | `src/app/api/internal/codex-responses-ws/route.ts` | Shared secret for the internal Codex Responses WebSocket bridge. Authenticates bridge requests between the Electron/browser WS relay and OmniRoute. ⚠️ **REQUIRED in production — when unset, all WS bridge requests are rejected.** Generate with `openssl rand -base64 32`. |
| `OMNIROUTE_PEER_STAMP_TOKEN` | No (auto) | _(auto per boot)_ | `src/server/authz/policies/management.ts` | Per-process secret proving the trusted peer-IP stamp came from OmniRoute's own HTTP server (`scripts/dev/peer-stamp.mjs`). The authz middleware trusts request locality (loopback/LAN gating of LOCAL_ONLY routes) only when the stamp carries this token. Auto-generated each boot — leave unset; only pin it for multi-process setups that must share the stamp. |
### Generation Commands
@@ -75,21 +76,27 @@ echo "OMNIROUTE_WS_BRIDGE_SECRET=$(openssl rand -base64 32)"
OmniRoute uses **SQLite** (via `better-sqlite3`) for all persistence. These variables control data location, encryption, and lifecycle.
| Variable | Default | Source File | Description |
| -------------------------------------- | -------------------- | ----------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| `DATA_DIR` | `~/.omniroute/` | `src/lib/db/core.ts` | Root directory for SQLite DB, backups, and data files. Override for Docker volumes or custom paths. |
| `STORAGE_ENCRYPTION_KEY` | _(empty = disabled)_ | `src/lib/db/encryption.ts` | AES key for full SQLite database encryption at rest. Generate with `openssl rand -hex 32`. |
| `STORAGE_ENCRYPTION_KEY_VERSION` | `v1` | `scripts/build/bootstrap-env.mjs`, `electron/main.js` | Version label for the encryption key. Increment when performing key rotation to support decryption of old backups. |
| `DISABLE_SQLITE_AUTO_BACKUP` | `false` | `src/lib/db/backup.ts` | When `true`, skips the automatic database backup that runs before migrations on every startup. |
| `OMNIROUTE_CRYPT_KEY` | _(unset)_ | `src/lib/db/encryption.ts` | **Legacy alias** for `STORAGE_ENCRYPTION_KEY`. Accepted as a fallback when the primary variable is absent. |
| `OMNIROUTE_API_KEY_BASE64` | _(unset)_ | `src/lib/db/encryption.ts` | **Legacy alias** (Base64-encoded form) accepted as a fallback. Decoded automatically before use. |
| `OMNIROUTE_DB_HEALTHCHECK_INTERVAL_MS` | _(unset)_ | `src/lib/db/core.ts` | Override the periodic SQLite healthcheck interval (ms). When unset, defaults are derived from `NODE_ENV`. |
| `OMNIROUTE_SKIP_DB_HEALTHCHECK` | `0` | `src/lib/db/core.ts`, `src/lib/db/healthCheck.ts` | Set to `1` to skip the DB healthcheck entirely on startup. Useful for short-lived tasks and integration tests. |
| `OMNIROUTE_FORCE_DB_HEALTHCHECK` | `0` | `src/lib/db/core.ts` | Set to `1` to force the DB healthcheck loop on, even when it would normally be skipped (e.g., short-lived tasks). |
| `OMNIROUTE_SKIP_POSTINSTALL` | `0` | `scripts/postinstall.mjs` | Set to `1` to skip the native-runtime warm-up during `npm install`. Useful in CI/headless installs where sqlite is already built. |
| `OMNIROUTE_MIGRATIONS_DIR` | _(auto-detect)_ | `src/lib/db/migrationRunner.ts` | Override the directory that the migration runner scans. Useful when shipping bundled migrations in custom builds. |
| `OMNIROUTE_SPEND_FLUSH_INTERVAL_MS` | _(default in code)_ | `src/lib/spend/batchWriter.ts` | Flush interval (ms) for the batched spend/cost writer. Lower values reduce write coalescing; higher values reduce DB contention. |
| `OMNIROUTE_SPEND_MAX_BUFFER_SIZE` | _(default in code)_ | `src/lib/spend/batchWriter.ts` | Max buffered spend entries before a forced flush. Raise on high-QPS deployments; lower when bounded memory matters more. |
| Variable | Default | Source File | Description |
| -------------------------------------- | -------------------- | ----------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `DATA_DIR` | `~/.omniroute/` | `src/lib/db/core.ts` | Root directory for SQLite DB, backups, and data files. Override for Docker volumes or custom paths. |
| `STORAGE_ENCRYPTION_KEY` | _(empty = disabled)_ | `src/lib/db/encryption.ts` | AES key for full SQLite database encryption at rest. Generate with `openssl rand -hex 32`. |
| `STORAGE_ENCRYPTION_KEY_VERSION` | `v1` | `scripts/build/bootstrap-env.mjs`, `electron/main.js` | Version label for the encryption key. Increment when performing key rotation to support decryption of old backups. |
| `DISABLE_SQLITE_AUTO_BACKUP` | `false` | `src/lib/db/backup.ts` | When `true`, skips the automatic database backup that runs before migrations on every startup. |
| `OMNIROUTE_CRYPT_KEY` | _(unset)_ | `src/lib/db/encryption.ts` | **Legacy alias** for `STORAGE_ENCRYPTION_KEY`. Accepted as a fallback when the primary variable is absent. |
| `OMNIROUTE_API_KEY_BASE64` | _(unset)_ | `src/lib/db/encryption.ts` | **Legacy alias** (Base64-encoded form) accepted as a fallback. Decoded automatically before use. |
| `OMNIROUTE_DB_HEALTHCHECK_INTERVAL_MS` | _(unset)_ | `src/lib/db/core.ts` | Override the periodic SQLite healthcheck interval (ms). When unset, defaults are derived from `NODE_ENV`. |
| `OMNIROUTE_SKIP_DB_HEALTHCHECK` | `0` | `src/lib/db/core.ts`, `src/lib/db/healthCheck.ts` | Set to `1` to skip the DB healthcheck entirely on startup. Useful for short-lived tasks and integration tests. |
| `OMNIROUTE_FORCE_DB_HEALTHCHECK` | `0` | `src/lib/db/core.ts` | Set to `1` to force the DB healthcheck loop on, even when it would normally be skipped (e.g., short-lived tasks). |
| `OMNIROUTE_SKIP_POSTINSTALL` | `0` | `scripts/postinstall.mjs` | Set to `1` to skip the native-runtime warm-up during `npm install`. Useful in CI/headless installs where sqlite is already built. |
| `OMNIROUTE_MIGRATIONS_DIR` | _(auto-detect)_ | `src/lib/db/migrationRunner.ts` | Override the directory that the migration runner scans. Useful when shipping bundled migrations in custom builds. |
| `OMNIROUTE_MAX_PENDING_MIGRATIONS` | `50` | `src/lib/db/migrationRunner.ts` | Mass-pending-migrations safety threshold (#3416). Startup aborts if more than this many migrations are pending on an existing DB (guards against a wiped tracking table). Raise it to restore an older backup; set to `0` to disable the check. |
| `OMNIROUTE_SPEND_FLUSH_INTERVAL_MS` | _(default in code)_ | `src/lib/spend/batchWriter.ts` | Flush interval (ms) for the batched spend/cost writer. Lower values reduce write coalescing; higher values reduce DB contention. |
| `OMNIROUTE_SPEND_MAX_BUFFER_SIZE` | _(default in code)_ | `src/lib/spend/batchWriter.ts` | Max buffered spend entries before a forced flush. Raise on high-QPS deployments; lower when bounded memory matters more. |
| `OMNIROUTE_PROXY_FETCH_DEBUG` | _(unset)_ | `open-sse/utils/proxyFetch.ts` | Set to `"true"` to emit `[ProxyFetch]` debug logs on the Vercel relay path. Off by default to avoid leaking routing hints. |
| `BATCH_RETRY_DURATION_MS` | `86400000` (24h) | `open-sse/services/batchProcessor.ts` | Maximum retry window for individual batch items (ms). Items exceeding this duration are marked failed. |
| `BATCH_BACKOFF_BASE_MS` | `5000` | `open-sse/services/batchProcessor.ts` | Base delay (ms) for exponential backoff on batch item retries. |
| `BATCH_BACKOFF_MAX_MS` | `3600000` (1h) | `open-sse/services/batchProcessor.ts` | Cap (ms) for exponential backoff between batch item retries. |
| `BATCH_MAX_CONCURRENT` | `1` | `open-sse/services/batchProcessor.ts` | Maximum number of batches processed concurrently. Raise to increase throughput; keep low to avoid rate-limit storms. |
### Scenarios
@@ -104,20 +111,29 @@ OmniRoute uses **SQLite** (via `better-sqlite3`) for all persistence. These vari
## 3. Network & Ports
| Variable | Default | Source File | Description |
| ------------------------------- | ------------------------------- | -------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `PORT` | `20128` | `src/lib/runtime/ports.ts` | Primary port for both Dashboard UI and API endpoints (single-port mode). |
| `API_PORT` | _(unset)_ | `src/lib/runtime/ports.ts` | When set, serves the `/v1/*` proxy API on this separate port. |
| `API_HOST` | `0.0.0.0` | `src/lib/runtime/ports.ts` | Bind address for the API port. |
| `DASHBOARD_PORT` | _(unset)_ | `src/lib/runtime/ports.ts` | When set, serves the Dashboard UI on this separate port. |
| `PROD_DASHBOARD_PORT` | `20130` | `docker-compose.prod.yml` | Host-side published port for the Dashboard in Docker production mode. |
| `PROD_API_PORT` | `20131` | `docker-compose.prod.yml` | Host-side published port for the API in Docker production mode. |
| `OMNIROUTE_PORT` | _(unset)_ | `src/lib/runtime/ports.ts` | Takes precedence over `PORT` when running inside Electron or other wrappers. |
| `NODE_ENV` | `production` | Next.js core | Controls logging verbosity, caching, error detail exposure, and Next.js optimizations. |
| `OMNIROUTE_USE_TURBOPACK` | `1` (default in `.env.example`) | `package.json` / Next.js 16 | Toggles the Next.js 16 Turbopack bundler in `npm run dev` and `npm run build`. Set to `0` on Windows or when running into native binding incompatibilities. |
| `OMNIROUTE_SKIP_DB_HEALTHCHECK` | _(unset)_ | `src/lib/db/core.ts` / `src/lib/db/healthCheck.ts` | Set to `1` to skip the SQLite integrity health check on startup. Useful for faster boot on large databases. |
| `HOST` | `0.0.0.0` | `scripts/dev/run-next.mjs` | Bind address for the Next.js dev/start server. Overrides the default `0.0.0.0` when set. |
| `HOSTNAME` | `127.0.0.1` | `scripts/dev/run-next-playwright.mjs` | Bind address used by the Playwright runner when launching Next.js. Defaults to `127.0.0.1` for hermetic tests. |
| Variable | Default | Source File | Description |
| ------------------------------------------- | ------------------------------- | ------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `PORT` | `20128` | `src/lib/runtime/ports.ts` | Primary port for both Dashboard UI and API endpoints (single-port mode). |
| `API_PORT` | _(unset)_ | `src/lib/runtime/ports.ts` | When set, serves the `/v1/*` proxy API on this separate port. |
| `API_HOST` | `0.0.0.0` | `src/lib/runtime/ports.ts` | Bind address for the API port. |
| `DASHBOARD_PORT` | _(unset)_ | `src/lib/runtime/ports.ts` | When set, serves the Dashboard UI on this separate port. |
| `PROD_DASHBOARD_PORT` | `20130` | `docker-compose.prod.yml` | Host-side published port for the Dashboard in Docker production mode. |
| `PROD_API_PORT` | `20131` | `docker-compose.prod.yml` | Host-side published port for the API in Docker production mode. |
| `OMNIROUTE_PORT` | _(unset)_ | `src/lib/runtime/ports.ts` | Takes precedence over `PORT` when running inside Electron or other wrappers. |
| `LIVE_WS_PORT` | `20129` | `src/server/ws/liveServer.ts` | Port for the real-time WebSocket live monitoring server. |
| `LIVE_WS_HOST` | `127.0.0.1` | `src/server/ws/liveServer.ts` | Bind address for the live WebSocket server. Set to `0.0.0.0` to expose on LAN (also configure `LIVE_WS_ALLOWED_ORIGINS`). |
| `LIVE_WS_ALLOWED_ORIGINS` | _(unset)_ | `src/server/ws/liveServer.ts` | Comma-separated extra origins allowed to open a live WebSocket. Loopback dashboard origins are already permitted by default. |
| `OMNIROUTE_ENABLE_LIVE_WS` | `true` | `src/server/ws/liveServer.ts` | Set to `0` or `false` to disable the real-time WebSocket server (enabled by default, loopback-bound). |
| `OMNIROUTE_DISABLE_LIVE_WS` | `false` | `scripts/start-ws-server.mjs` | CI/harness toggle that disables the standalone live WebSocket helper script. |
| `RELAY_IP_PER_MINUTE` | `30` | `src/app/api/v1/relay/chat/completions/route.ts` | Per-(token, IP) relay rate limit, requests/minute. In-memory, per instance. `0` or negative disables the IP-dimension gate (per-token DB limit still applies). |
| `NODE_ENV` | `production` | Next.js core | Controls logging verbosity, caching, error detail exposure, and Next.js optimizations. |
| `OMNIROUTE_USE_TURBOPACK` | `1` (default in `.env.example`) | `package.json` / Next.js 16 | Toggles the Next.js 16 Turbopack bundler in `npm run dev` and `npm run build`. Set to `0` on Windows or when running into native binding incompatibilities. |
| `OMNIROUTE_SKIP_DB_HEALTHCHECK` | _(unset)_ | `src/lib/db/core.ts` / `src/lib/db/healthCheck.ts` | Set to `1` to skip the SQLite integrity health check on startup. Useful for faster boot on large databases. |
| `CREDENTIAL_HEALTH_CHECK_INTERVAL` | `300000` | `open-sse/config/constants.ts` / `src/lib/credentialHealth/scheduler.ts` | Interval (ms) for the background credential health check scheduler. Minimum: 10000 (10s). |
| `CREDENTIAL_HEALTH_CACHE_TTL` | `300000` | `open-sse/config/constants.ts` / `src/lib/credentialHealth/cache.ts` | TTL (ms) for cached credential health status. |
| `OMNIROUTE_DISABLE_CREDENTIAL_HEALTH_CHECK` | `false` | `src/lib/credentialHealth/scheduler.ts` | Set to `1` or `true` to disable background periodic testing of provider connections. |
| `HOST` | `0.0.0.0` | `scripts/dev/run-next.mjs` | Bind address for the Next.js dev/start server. Overrides the default `0.0.0.0` when set. |
| `HOSTNAME` | `127.0.0.1` | `scripts/dev/run-next-playwright.mjs` | Bind address used by the Playwright runner when launching Next.js. Defaults to `127.0.0.1` for hermetic tests. |
### Port Modes
@@ -206,11 +222,12 @@ OmniRoute provides a two-layer defense: request-side injection scanning and resp
## 6. Tool & Routing Policies
| Variable | Default | Source File | Description |
| ----------------------------------- | ---------------------------- | ----------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| `TOOL_POLICY_MODE` | `disabled` | `src/lib/toolPolicy.ts` | Controls LLM tool/function-calling access. `allowlist` = only listed tools, `denylist` = all except listed, `disabled` = no restrictions. |
| `OMNIROUTE_PAYLOAD_RULES_PATH` | `./config/payloadRules.json` | `open-sse/services/payloadRules.ts` | Path to payload manipulation rules JSON file (per-model/protocol upstream tweaks). |
| `OMNIROUTE_PAYLOAD_RULES_RELOAD_MS` | `5000` | `open-sse/services/payloadRules.ts` | Reload interval (ms) for hot-reloading the payload rules file. Minimum `1000`. |
| Variable | Default | Source File | Description |
| ----------------------------------------------------------- | ---------------------------- | ----------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `TOOL_POLICY_MODE` | `disabled` | `src/lib/toolPolicy.ts` | Controls LLM tool/function-calling access. `allowlist` = only listed tools, `denylist` = all except listed, `disabled` = no restrictions. |
| `OMNIROUTE_PAYLOAD_RULES_PATH` | `./config/payloadRules.json` | `open-sse/services/payloadRules.ts` | Path to payload manipulation rules JSON file (per-model/protocol upstream tweaks). |
| `OMNIROUTE_PAYLOAD_RULES_RELOAD_MS` | `5000` | `open-sse/services/payloadRules.ts` | Reload interval (ms) for hot-reloading the payload rules file. Minimum `1000`. |
| `OMNIROUTE_PREFER_CLAUDE_CODE_FOR_UNPREFIXED_CLAUDE_MODELS` | `false` | `open-sse/services/model.ts` | Opt-in: route bare `claude-*` model IDs from Claude Code clients through the Claude Code OAuth account instead of requiring a provider prefix. Explicit provider prefixes still win. Also configurable via a dashboard toggle on the Claude provider page. |
---
@@ -221,17 +238,24 @@ OmniRoute provides a two-layer defense: request-side injection scanning and resp
| `BASE_URL` | `http://localhost:20128` | `src/lib/cloudSync.ts` | Server-side URL for internal sync jobs to call `/api/sync/cloud`. |
| `CLOUD_URL` | _(empty)_ | `src/lib/cloudSync.ts` | Cloud relay endpoint URL (premium feature). |
| `CLOUD_SYNC_TIMEOUT_MS` | `12000` | `src/lib/cloudSync.ts` | HTTP timeout for cloud sync requests. |
| `OMNIROUTE_BUILD_PROFILE` | `full` | Webpack build config | Build-time profile (set to `minimal` to physically exclude privileged modules from bundle). |
| `OMNIROUTE_CLOUD_SYNC_SECRET` | _(empty)_ | `src/lib/cloudSync.ts` | Shared secret used to verify the HMAC-SHA256 signature of Cloud Sync responses. |
| `OMNIROUTE_CLOUD_SYNC_SECRETS` | `false` | `src/lib/cloudSync.ts` | Set to `true` to allow the Cloud Sync endpoint to overwrite local credentials. Default is `false`. |
| `OMNIROUTE_ZED_IMPORT_LEGACY_ONE_STEP` | `false` | `src/app/api/providers/zed/import/route.ts` | Set to `true` to fall back to the v3.8.5 one-step "import everything" behavior without user confirmation. |
| `NEXT_PUBLIC_BASE_URL` | `http://localhost:20128` | OAuth, Dashboard, sync | Public-facing URL for OAuth redirect_uri, Dashboard links. **Must match your public URL behind reverse proxy.** |
| `NEXT_PUBLIC_CLOUD_URL` | _(empty)_ | Client-side | Client-side mirror of `CLOUD_URL`. |
| `NEXT_PUBLIC_APP_URL` | _(unset)_ | `src/shared/services/cloudSyncScheduler.ts` | Legacy fallback for `NEXT_PUBLIC_BASE_URL`. |
| `OMNIROUTE_PUBLIC_BASE_URL` | _(unset)_ | `open-sse/executors/chatgpt-web.ts` | Browser-facing OmniRoute origin used for image URLs in API responses (e.g., `/v1/chatgpt-web/image/<id>`). Set this when OpenWebUI or another relay reaches OmniRoute by an internal URL but the user's browser must fetch images from a LAN, tunnel, or public origin. Do **not** include `/v1`. |
| `OMNIROUTE_CGPT_WEB_IMAGE_TIMEOUT_MS` | `180000` (3 min) | `open-sse/executors/chatgpt-web.ts` | Max wait time for an async chatgpt-web image to land via the celsius WebSocket. Increase during upstream queue-deep windows. |
| `OMNIROUTE_CGPT_WEB_IMAGE_CACHE_MAX_MB` | `256` | `open-sse/services/chatgptImageCache.ts` | Total in-memory byte budget (MB) for the chatgpt-web image cache serving `/v1/chatgpt-web/image/<id>`. Lower on memory-constrained hosts; raise if image generation is heavy and clients race the 30-minute TTL. |
| `THEOLDLLM_NAV_TIMEOUT_MS` | `30000` (30s) | `open-sse/executors/theoldllm.ts` | Playwright navigation timeout (ms) for the browser-backed token capture used by the The Old LLM (theoldllm) free provider. Raise on slow networks if the relay page is slow to settle. |
| `KIE_CALLBACK_URL` | _(unset)_ | `open-sse/utils/kieTask.ts` | Public callback URL for asynchronous kie.ai jobs. Highest-priority override before `OMNIROUTE_KIE_CALLBACK_URL` and `OMNIROUTE_PUBLIC_URL`. |
| `OMNIROUTE_KIE_CALLBACK_URL` | _(unset)_ | `open-sse/utils/kieTask.ts` | Alternate spelling of `KIE_CALLBACK_URL`. Falls back when the primary variable is unset. |
| `OMNIROUTE_PUBLIC_URL` | _(unset)_ | `open-sse/utils/kieTask.ts` | Public origin used to compose async callback URLs. Lowest-priority fallback for kie.ai callbacks; also used as a generic public URL for other relays. |
| `OMNIROUTE_CROF_USAGE_URL` | `https://crof.ai/usage_api/` | `open-sse/services/usage.ts` | CrofAI quota lookup endpoint used by the Usage page. Override for relays / test fixtures. |
| `OMNIROUTE_GEMINI_CLI_USAGE_URL` | `https://cloudcode-pa.googleapis.com/v1internal:loadCodeAssist` | `open-sse/services/usage.ts` | Gemini CLI quota lookup endpoint. Override for relays / test fixtures. |
| `OMNIROUTE_OPENCODE_QUOTA_URL` | `https://opencode.ai/zen/go/v1/quota` | `open-sse/services/opencodeQuotaFetcher.ts` | OpenCode (zen/go) quota lookup endpoint used by the Usage page. Override for relays / test fixtures. |
| `OMNIROUTE_OPENCODE_GO_QUOTA_URL` | `https://api.z.ai/api/monitor/usage/quota/limit` | `open-sse/services/usage.ts` | OpenCode Go quota lookup endpoint used by the Usage page. Override for relays / test fixtures. |
| `OMNIROUTE_CODEWHISPERER_BASE_URL` | `https://codewhisperer.us-east-1.amazonaws.com` | `open-sse/services/usage.ts` | CodeWhisperer (AWS Kiro) usage limits endpoint. Override for relays / test fixtures. |
> [!IMPORTANT]
@@ -243,23 +267,34 @@ OmniRoute provides a two-layer defense: request-side injection scanning and resp
Route upstream LLM provider calls through an HTTP or SOCKS5 proxy for egress control, geo-routing, or IP masking.
| Variable | Default | Source File | Description |
| --------------------------------- | --------- | -------------------- | ----------------------------------------------------------------------------------- |
| `ENABLE_SOCKS5_PROXY` | `true` | `open-sse/executors` | Enable SOCKS5 proxy agent for upstream calls. |
| `NEXT_PUBLIC_ENABLE_SOCKS5_PROXY` | `true` | Client-side | Client-side awareness of SOCKS5 availability. |
| `HTTP_PROXY` | _(unset)_ | Node.js standard | HTTP proxy for upstream calls. |
| `HTTPS_PROXY` | _(unset)_ | Node.js standard | HTTPS proxy for upstream calls. |
| `ALL_PROXY` | _(unset)_ | Node.js standard | Universal proxy (supports `socks5://`). |
| `NO_PROXY` | _(unset)_ | Node.js standard | Comma-separated hostnames/IPs to bypass the proxy. |
| `ENABLE_TLS_FINGERPRINT` | `false` | `open-sse/executors` | Spoof TLS fingerprint using wreq-js (mimics Chrome 124). Counters JA3/JA4 blocking. |
| Variable | Default | Source File | Description |
| --------------------------------------- | --------- | -------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `ENABLE_SOCKS5_PROXY` | `true` | `open-sse/executors` | Enable SOCKS5 proxy agent for upstream calls. Opt-out with `false`. |
| `NEXT_PUBLIC_ENABLE_SOCKS5_PROXY` | `true` | Client-side | Client-side awareness of SOCKS5 availability. |
| `HTTP_PROXY` | _(unset)_ | Node.js standard | HTTP proxy for upstream calls. |
| `HTTPS_PROXY` | _(unset)_ | Node.js standard | HTTPS proxy for upstream calls. |
| `ALL_PROXY` | _(unset)_ | Node.js standard | Universal proxy (supports `socks5://`). |
| `NO_PROXY` | _(unset)_ | Node.js standard | Comma-separated hostnames/IPs to bypass the proxy. |
| `PROXY_FAIL_OPEN` | `false` | `src/sse/handlers/chatHelpers.ts` | When `false` (default), a request whose assigned proxy fails to resolve is **refused (fail-closed)** rather than falling back to a direct connection — prevents real-IP leaks. Set `true` to restore the legacy DIRECT fallback. |
| `ENABLE_TLS_FINGERPRINT` | `false` | `open-sse/executors` | Spoof TLS fingerprint using wreq-js (mimics Chrome 124). Counters JA3/JA4 blocking. |
| `OMNIROUTE_TURNSTILE_IGNORE_TLS_ERRORS` | `false` | `open-sse/services/claudeTurnstileSolver.ts` | Allow the Claude Turnstile Playwright browser context to ignore HTTPS certificate errors. |
### Scenarios
| Scenario | Configuration |
| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| **SOCKS5 through SSH tunnel** | `ALL_PROXY=socks5://127.0.0.1:7890`, `ENABLE_SOCKS5_PROXY=true` |
| **Corporate HTTP proxy** | `HTTP_PROXY=http://proxy.corp.com:3128`, `HTTPS_PROXY=http://proxy.corp.com:3128`, `NO_PROXY=localhost,internal.corp.com` |
| **Anti-fingerprint** | `ENABLE_TLS_FINGERPRINT=true` — requires `wreq-js` (included) |
| Scenario | Configuration |
| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| **SOCKS5 through SSH tunnel** | `ALL_PROXY=socks5://127.0.0.1:7890`, `ENABLE_SOCKS5_PROXY=true` |
| **Corporate HTTP proxy** | `HTTP_PROXY=http://proxy.corp.com:3128`, `HTTPS_PROXY=http://proxy.corp.com:3128`, `NO_PROXY=localhost,internal.corp.com` |
| **Anti-fingerprint** | `ENABLE_TLS_FINGERPRINT=true` — requires `wreq-js` (included) |
| **Egress-controlled / no direct access** | Leave `PROXY_FAIL_OPEN=false` (default). Requests fail hard when the proxy is unavailable instead of leaking via direct. |
| **Legacy / dev — allow direct fallback** | `PROXY_FAIL_OPEN=true`. Restores pre-hardening behaviour: direct connection used when proxy resolution fails. |
> **Note (NVIDIA validation bypass — #3226):** NVIDIA's API-key validation endpoint
> stalls when routed through the global proxy/TLS-patched fetch (undici dispatcher → 504).
> `src/lib/providers/validation.ts::directHttpsRequest()` intentionally bypasses the
> proxy patch for that one validation call using `safeOutboundFetch({ bypassProxyPatch: true })`.
> This is a documented, scoped exception — it does **not** affect chat/usage egress.
> The bypass is scope-pinned by `tests/unit/proxy-bypass-scope-guard-3226.test.ts`.
---
@@ -267,22 +302,23 @@ Route upstream LLM provider calls through an HTTP or SOCKS5 proxy for egress con
Controls how OmniRoute discovers and launches CLI sidecars (Claude Code, Codex, etc.).
| Variable | Default | Source File | Description |
| ------------------------- | ---------- | ----------------------------------- | ---------------------------------------------------------------------------------- |
| `CLI_MODE` | `auto` | `src/shared/services/cliRuntime.ts` | `auto` = search system PATH; `manual` = use explicit paths only. |
| `CLI_EXTRA_PATHS` | _(unset)_ | `src/shared/services/cliRuntime.ts` | Additional PATH entries for CLI binary discovery (colon-separated). |
| `CLI_CONFIG_HOME` | _(unset)_ | `src/shared/services/cliRuntime.ts` | Override home directory for reading CLI configs (`~/.claude`, `~/.codex`). |
| `CLI_ALLOW_CONFIG_WRITES` | `false` | `src/shared/services/cliRuntime.ts` | Allow OmniRoute to write CLI config files (token refresh, session data). |
| `CLI_CLAUDE_BIN` | `claude` | `src/shared/services/cliRuntime.ts` | Custom path to Claude CLI binary. |
| `CLI_CODEX_BIN` | `codex` | `src/shared/services/cliRuntime.ts` | Custom path to Codex CLI binary. |
| `CLI_DROID_BIN` | `droid` | `src/shared/services/cliRuntime.ts` | Custom path to Droid CLI binary. |
| `CLI_OPENCLAW_BIN` | `openclaw` | `src/shared/services/cliRuntime.ts` | Custom path to OpenClaw CLI binary. |
| `CLI_CURSOR_BIN` | `agent` | `src/shared/services/cliRuntime.ts` | Custom path to Cursor agent binary. |
| `CLI_CLINE_BIN` | `cline` | `src/shared/services/cliRuntime.ts` | Custom path to Cline CLI binary. |
| `CLI_CONTINUE_BIN` | `cn` | `src/shared/services/cliRuntime.ts` | Custom path to Continue CLI binary. |
| `CLI_QODER_BIN` | `qoder` | `src/shared/services/cliRuntime.ts` | Custom path to Qoder CLI binary. |
| `CLI_QWEN_BIN` | `qwen` | `src/shared/services/cliRuntime.ts` | Custom path to the Qwen Code CLI binary. |
| `CLI_DEVIN_BIN` | `devin` | `open-sse/executors/devin-cli.ts` | Custom path to the Devin CLI binary (v3.8.0). Used by the Windsurf/Devin executor. |
| Variable | Default | Source File | Description |
| ------------------------- | ----------- | --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `CLI_MODE` | `auto` | `src/shared/services/cliRuntime.ts` | `auto` = search system PATH; `manual` = use explicit paths only. |
| `CLI_EXTRA_PATHS` | _(unset)_ | `src/shared/services/cliRuntime.ts` | Additional PATH entries for CLI binary discovery (colon-separated). |
| `CLI_CONFIG_HOME` | _(unset)_ | `src/shared/services/cliRuntime.ts` | Override home directory for reading CLI configs (`~/.claude`, `~/.codex`). |
| `CLI_ALLOW_CONFIG_WRITES` | `false` | `src/shared/services/cliRuntime.ts` | Allow OmniRoute to write CLI config files (token refresh, session data). |
| `CLI_CLAUDE_BIN` | `claude` | `src/shared/services/cliRuntime.ts` | Custom path to Claude CLI binary. |
| `CLI_CODEX_BIN` | `codex` | `src/shared/services/cliRuntime.ts` | Custom path to Codex CLI binary. |
| `CLI_DROID_BIN` | `droid` | `src/shared/services/cliRuntime.ts` | Custom path to Droid CLI binary. |
| `CLI_OPENCLAW_BIN` | `openclaw` | `src/shared/services/cliRuntime.ts` | Custom path to OpenClaw CLI binary. |
| `CLI_CURSOR_BIN` | `agent` | `src/shared/services/cliRuntime.ts` | Custom path to Cursor agent binary. |
| `CLI_CLINE_BIN` | `cline` | `src/shared/services/cliRuntime.ts` | Custom path to Cline CLI binary. |
| `CLI_CONTINUE_BIN` | `cn` | `src/shared/services/cliRuntime.ts` | Custom path to Continue CLI binary. |
| `CLI_QODER_BIN` | `qoder` | `src/shared/services/cliRuntime.ts` | Custom path to Qoder CLI binary. |
| `CLI_QWEN_BIN` | `qwen` | `src/shared/services/cliRuntime.ts` | Custom path to the Qwen Code CLI binary. |
| `CLI_DEVIN_BIN` | `devin` | `open-sse/executors/devin-cli.ts` | Custom path to the Devin CLI binary (v3.8.0). Used by the Windsurf/Devin executor. |
| `HERMES_HOME` | `~/.hermes` | `src/lib/cli-helper/config-generator/hermesHome.ts` | Hermes Agent home directory where OmniRoute reads/writes the Hermes CLI config. Matches the env var the Hermes PowerShell installer sets on Windows (`%LOCALAPPDATA%\hermes`). |
### Docker Example
@@ -299,41 +335,45 @@ CLI_CLAUDE_BIN=/host-cli/bin/claude
These variables tune the `omniroute` CLI binary's own behavior (not the sidecar
detection above).
| Variable | Default | Source File | Description |
| --------------------------- | ---------- | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| `OMNIROUTE_LANG` | _(system)_ | `bin/cli/i18n.mjs` | Force CLI output language. BCP-47 locale (e.g. `en`, `pt-BR`). Overrides system locale env vars (LC_ALL, LC_MESSAGES). |
| `OMNIROUTE_SHOW_LOG` | _(unset)_ | `bin/cli/runtime/processSupervisor.mjs` | Set to `1` to forward server stdout/stderr to the terminal in supervised mode. Equivalent to `--log` flag on `omniroute serve`. |
| `OMNIROUTE_CLI_TOKEN` | _(unset)_ | `bin/cli/api.mjs` | Machine-auth token injected as `x-omniroute-cli-token` header. Auto-generated in task 8.12. |
| `OMNIROUTE_HTTP_TIMEOUT_MS` | `30000` | `bin/cli/api.mjs` | Per-attempt HTTP timeout (ms) for CLI → server requests. |
| `OMNIROUTE_VERBOSE` | `0` | `bin/cli/api.mjs` | Set to `1` to print retry/backoff diagnostics to stderr during CLI commands. |
| `OMNIROUTE_PLUGIN_PATH` | _(unset)_ | `bin/cli/plugins.mjs` | Custom directory for CLI plugin discovery (`omniroute-cmd-*` packages). Defaults to `~/.omniroute/plugins/` when unset. |
| Variable | Default | Source File | Description |
| ------------------------------ | ---------- | --------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
| `OMNIROUTE_LANG` | _(system)_ | `bin/cli/i18n.mjs` | Force CLI output language. BCP-47 locale (e.g. `en`, `pt-BR`). Overrides system locale env vars (LC_ALL, LC_MESSAGES). |
| `OMNIROUTE_SHOW_LOG` | _(unset)_ | `bin/cli/runtime/processSupervisor.mjs` | Set to `1` to forward server stdout/stderr to the terminal in supervised mode. Equivalent to `--log` flag on `omniroute serve`. |
| `OMNIROUTE_CLI_TOKEN` | _(unset)_ | `bin/cli/api.mjs` | Machine-auth token injected as `x-omniroute-cli-token` header. Auto-generated in task 8.12. |
| `OMNIROUTE_HTTP_TIMEOUT_MS` | `30000` | `bin/cli/api.mjs` | Per-attempt HTTP timeout (ms) for CLI → server requests. |
| `OMNIROUTE_VERBOSE` | `0` | `bin/cli/api.mjs` | Set to `1` to print retry/backoff diagnostics to stderr during CLI commands. |
| `OMNIROUTE_PLUGIN_PATH` | _(unset)_ | `bin/cli/plugins.mjs` | Custom directory for CLI plugin discovery (`omniroute-cmd-*` packages). Defaults to `~/.omniroute/plugins/` when unset. |
| `OMNIROUTE_PLUGINS_ALLOW_EXEC` | `0` | `src/lib/plugins/pluginWorker.ts` | Set to `1` to allow plugins to request the `exec` permission (spawn child processes from the worker sandbox). Local operator only. |
---
## 10. Internal Agent & MCP Integrations
| Variable | Default | Source File | Description |
| ----------------------------------------------- | ----------- | ----------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| `OMNIROUTE_BASE_URL` | auto-detect | `open-sse/mcp-server/server.ts` | Explicit URL for MCP/A2A tools to reach OmniRoute. Overrides localhost auto-detection. |
| `OMNIROUTE_API_KEY` | _(unset)_ | MCP/A2A modules | API key for internal MCP tool and A2A skill calls. |
| `OMNIROUTE_API_KEY_ID` | _(unset)_ | `open-sse/mcp-server/audit.ts` | Key ID for MCP audit log attribution. |
| `ROUTER_API_KEY` | _(unset)_ | Legacy | Legacy alias for `OMNIROUTE_API_KEY`. |
| `OMNIROUTE_MCP_ENFORCE_SCOPES` | `false` | `open-sse/mcp-server/server.ts` | Enforce scope-based access control on MCP tool calls. |
| `OMNIROUTE_MCP_SCOPES` | _(all)_ | `open-sse/mcp-server/server.ts` | Comma-separated scopes: `admin`, `combos`, `health`, `models`, `routing`, `budget`, `metrics`, `pricing`, `memory`, `skills`. |
| `OMNIROUTE_MCP_COMPRESS_DESCRIPTIONS` | enabled | `open-sse/mcp-server/descriptionCompressor.ts` | Compress MCP tool descriptions before serializing the manifest. Disable values: `0`, `false`, `off`. |
| `OMNIROUTE_MCP_DESCRIPTION_COMPRESSION` | `rtk` | `open-sse/mcp-server/descriptionCompressor.ts` | Compression algorithm/profile. Disable values: `0`, `false`, `off`. |
| `MODEL_SYNC_INTERVAL_HOURS` | `24` | `src/shared/services/modelSyncScheduler.ts` | Model catalog sync interval in hours. |
| `PROVIDER_LIMITS_SYNC_INTERVAL_MINUTES` | `70` | `src/server-init.ts` | Provider rate-limit and quota polling interval. |
| `OMNIROUTE_DISABLE_BACKGROUND_SERVICES` | `false` | `src/instrumentation-node.ts` | Disable all background services (sync, pricing, model refresh). Useful for CI/test. |
| `OMNIROUTE_ENABLE_RUNTIME_BACKGROUND_TASKS` | _(unset)_ | `src/lib/config/runtimeSettings.ts` | Force background tasks on under automated test detection. Set `1` to override the test heuristic. |
| `OMNIROUTE_BUDGET_RESET_JOB_INTERVAL_MS` | `600000` | `src/lib/jobs/budgetResetJob.ts` | Budget reset check cadence (ms). Floor `10000`. |
| `OMNIROUTE_REASONING_CACHE_CLEANUP_INTERVAL_MS` | `1800000` | `src/lib/jobs/reasoningCacheCleanupJob.ts` | Reasoning cache cleanup cadence (ms). Floor `60000`. |
| `OMNIROUTE_CONFIG_HOT_RELOAD_MS` | `5000` | `src/lib/config/hotReload.ts` | Polling interval (ms) for config hot-reload. Lower than `1000` is rejected. |
| `OMNIROUTE_DISABLE_REDIS_AUTH_CACHE` | _(enabled)_ | `src/lib/db/apiKeys.ts` | Set `1` to bypass the Redis-backed API-key auth cache (forces DB reads). |
| `OMNIROUTE_RTK_TRUST_PROJECT_FILTERS` | `0` | `open-sse/services/compression/engines/rtk/filterLoader.ts` | Trust user-managed RTK project filter rules without strict signature checks. |
| `OMNIROUTE_BOOTSTRAPPED` | `false` | `src/app/(dashboard)/dashboard/page.tsx` | Set `true` by bootstrap script after initial setup. Controls setup wizard visibility. |
| `OMNIROUTE_ALLOW_BODY_PROJECT_OVERRIDE` | `0` | `open-sse/executors/antigravity.ts` | Escape hatch: allow request body to override the Antigravity project field. |
| `ANTIGRAVITY_CREDITS` | _(unset)_ | `open-sse/services/antigravityCredits.ts` | Override Antigravity's advertised remaining credits (testing / forced values). |
| Variable | Default | Source File | Description |
| ----------------------------------------------- | --------------------------------------------------- | ----------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `OMNIROUTE_BASE_URL` | auto-detect | `open-sse/mcp-server/server.ts` | Explicit URL for MCP/A2A tools to reach OmniRoute. Overrides localhost auto-detection. |
| `OMNIROUTE_API_KEY` | _(unset)_ | MCP/A2A modules | API key for internal MCP tool and A2A skill calls. |
| `OMNIROUTE_API_KEY_ID` | _(unset)_ | `open-sse/mcp-server/audit.ts` | Key ID for MCP audit log attribution. |
| `ROUTER_API_KEY` | _(unset)_ | Legacy | Legacy alias for `OMNIROUTE_API_KEY`. |
| `OMNIROUTE_MCP_ENFORCE_SCOPES` | `false` | `open-sse/mcp-server/server.ts` | Enforce scope-based access control on MCP tool calls. |
| `OMNIROUTE_MCP_SCOPES` | _(all)_ | `open-sse/mcp-server/server.ts` | Comma-separated scopes: `admin`, `combos`, `health`, `models`, `routing`, `budget`, `metrics`, `pricing`, `memory`, `skills`. |
| `OMNIROUTE_MCP_COMPRESS_DESCRIPTIONS` | enabled | `open-sse/mcp-server/descriptionCompressor.ts` | Compress MCP tool descriptions before serializing the manifest. Disable values: `0`, `false`, `off`. |
| `OMNIROUTE_MCP_DESCRIPTION_COMPRESSION` | `rtk` | `open-sse/mcp-server/descriptionCompressor.ts` | Compression algorithm/profile. Disable values: `0`, `false`, `off`. |
| `MODEL_SYNC_INTERVAL_HOURS` | `24` | `src/shared/services/modelSyncScheduler.ts` | Model catalog sync interval in hours. |
| `PROVIDER_LIMITS_SYNC_INTERVAL_MINUTES` | `70` | `src/server-init.ts` | Provider rate-limit and quota polling interval. |
| `PROVIDER_LIMITS_SYNC_SPACING_MS` | `1500` | `src/lib/usage/providerLimits.ts` | Gap (ms) between consecutive OAuth quota fetches in a bulk sync; OAuth connections are fetched one at a time to avoid bursting an upstream. `0` opts out (concurrent). |
| `PROVIDER_LIMITS_POST_USAGE_REFRESH_DELAY_MS` | `5000` | `src/lib/usage/providerLimits.ts` | Delay (ms) before refreshing provider limits after a real usage event, giving the upstream quota API time to register consumption. |
| `OMNIROUTE_DISABLE_BACKGROUND_SERVICES` | `false` | `src/instrumentation-node.ts` | Disable all background services (sync, pricing, model refresh). Useful for CI/test. |
| `OMNIROUTE_ENABLE_RUNTIME_BACKGROUND_TASKS` | _(unset)_ | `src/lib/config/runtimeSettings.ts` | Force background tasks on under automated test detection. Set `1` to override the test heuristic. |
| `OMNIROUTE_BUDGET_RESET_JOB_INTERVAL_MS` | `600000` | `src/lib/jobs/budgetResetJob.ts` | Budget reset check cadence (ms). Floor `10000`. |
| `OMNIROUTE_REASONING_CACHE_CLEANUP_INTERVAL_MS` | `1800000` | `src/lib/jobs/reasoningCacheCleanupJob.ts` | Reasoning cache cleanup cadence (ms). Floor `60000`. |
| `OMNIROUTE_CONFIG_HOT_RELOAD_MS` | `5000` | `src/lib/config/hotReload.ts` | Polling interval (ms) for config hot-reload. Lower than `1000` is rejected. |
| `OMNIROUTE_DISABLE_REDIS_AUTH_CACHE` | _(enabled)_ | `src/lib/db/apiKeys.ts` | Set `1` to bypass the Redis-backed API-key auth cache (forces DB reads). |
| `OMNIROUTE_RTK_TRUST_PROJECT_FILTERS` | `0` | `open-sse/services/compression/engines/rtk/filterLoader.ts` | Trust user-managed RTK project filter rules without strict signature checks. |
| `OMNIROUTE_BOOTSTRAPPED` | `false` | `src/app/(dashboard)/dashboard/page.tsx` | Set `true` by bootstrap script after initial setup. Controls setup wizard visibility. |
| `OMNIROUTE_ALLOW_BODY_PROJECT_OVERRIDE` | `0` | `open-sse/executors/antigravity.ts` | Escape hatch: allow request body to override the Antigravity project field. |
| `ANTIGRAVITY_CREDITS` | _(unset)_ | `open-sse/services/antigravityCredits.ts` | Override Antigravity's advertised remaining credits (testing / forced values). |
| `AGY_TOKEN_FILE` | `~/.gemini/antigravity-cli/antigravity-oauth-token` | `src/app/api/providers/agy-auth/apply-local/route.ts` | Override the Antigravity CLI (agy) token-file path for the auto-detect local login import. |
### OAuth CLI Bridge (Internal)
@@ -407,18 +447,20 @@ process.env[`${PROVIDER_ID}_USER_AGENT`]
> **Source:** `open-sse/executors/base.ts` → `buildHeaders()`
| Variable | Default Value | When to Update |
| ------------------------ | --------------------------------------------- | ------------------------------------------------------------- |
| `CLAUDE_USER_AGENT` | `claude-cli/2.1.145 (external, cli)` | When Anthropic releases a new CLI version |
| `CODEX_USER_AGENT` | `codex-cli/0.132.0 (Windows 10.0.26200; x64)` | When OpenAI updates the Codex CLI |
| `CODEX_CLIENT_VERSION` | `0.131.0` | Override Codex client version independently of full UA string |
| `GITHUB_USER_AGENT` | `GitHubCopilotChat/0.45.1` | When GitHub Copilot Chat updates |
| `ANTIGRAVITY_USER_AGENT` | `antigravity/2.0.1 darwin/arm64` | When Antigravity IDE updates |
| `KIRO_USER_AGENT` | `AWS-SDK-JS/3.0.0 kiro-ide/1.0.0` | When Kiro IDE updates |
| `QODER_USER_AGENT` | `Qoder-Cli` | When Qoder CLI updates |
| `QWEN_USER_AGENT` | `QwenCode/0.15.9 (linux; x64)` | When Qwen Code updates |
| `CURSOR_USER_AGENT` | `Cursor/3.3` | When Cursor updates |
| `GEMINI_CLI_USER_AGENT` | `google-api-nodejs-client/10.3.0` | When Google API client updates |
| Variable | Default Value | When to Update |
| -------------------------------- | --------------------------------------------- | ------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `CLAUDE_USER_AGENT` | `claude-cli/2.1.145 (external, cli)` | When Anthropic releases a new CLI version |
| `CLAUDE_DISABLE_TOOL_NAME_CLOAK` | `false` | `executors/base.ts` + `executors/cliproxyapi.ts` | Set to `1`/`true` to forward third-party harness tool names verbatim to Anthropic on both Anthropic-bound paths (native OAuth and CLIProxyAPI). By default the executor deterministically aliases non-Claude-Code tool names (Claude Code canonical mapping where one exists, otherwise PascalCase) and reverses them on the response via `_toolNameMap`, so harnesses with snake_case tools are not refused as fingerprinted third-party clients. Debugging only. |
| `CODEX_USER_AGENT` | `codex-cli/0.132.0 (Windows 10.0.26200; x64)` | When OpenAI updates the Codex CLI |
| `CODEX_CLIENT_VERSION` | `0.131.0` | Override Codex client version independently of full UA string |
| `GITHUB_USER_AGENT` | `GitHubCopilotChat/0.45.1` | When GitHub Copilot Chat updates |
| `ANTIGRAVITY_USER_AGENT` | `antigravity/2.0.1 darwin/arm64` | When Antigravity IDE updates |
| `KIRO_USER_AGENT` | `AWS-SDK-JS/3.0.0 kiro-ide/1.0.0` | When Kiro IDE updates |
| `KIRO_OAUTH_CLIENT_ID` | `kiro-cli` | Override the Kiro social device-code `clientId` (public id) |
| `QODER_USER_AGENT` | `Qoder-Cli` | When Qoder CLI updates |
| `QWEN_USER_AGENT` | `QwenCode/0.15.9 (linux; x64)` | When Qwen Code updates |
| `CURSOR_USER_AGENT` | `Cursor/3.3` | When Cursor updates |
| `GEMINI_CLI_USER_AGENT` | `google-api-nodejs-client/10.3.0` | When Google API client updates |
> [!TIP]
> You can add User-Agent overrides for **any** provider using the pattern `{PROVIDER_ID}_USER_AGENT`. The executor dynamically constructs the env var name.
@@ -499,6 +541,7 @@ REQUEST_TIMEOUT_MS (global override)
│ ├── FETCH_CONNECT_TIMEOUT_MS (independent, default: 30000)
│ └── FETCH_KEEPALIVE_TIMEOUT_MS (independent, default: 4000)
├─→ STREAM_IDLE_TIMEOUT_MS (inherits from REQUEST_TIMEOUT_MS, default: 600000)
├─→ STREAM_READINESS_TIMEOUT_MS (inherits from REQUEST_TIMEOUT_MS, default: 80000)
└─→ API_BRIDGE_PROXY_TIMEOUT_MS (inherits from REQUEST_TIMEOUT_MS, default: 30000)
├─→ API_BRIDGE_SERVER_REQUEST_TIMEOUT_MS (derived, default: 300000)
├── API_BRIDGE_SERVER_HEADERS_TIMEOUT_MS (default: 60000)
@@ -506,29 +549,39 @@ REQUEST_TIMEOUT_MS (global override)
└── API_BRIDGE_SERVER_SOCKET_TIMEOUT_MS (default: 0 = disabled)
```
| Variable | Default | Description |
| ---------------------------------------- | -------------------- | ------------------------------------------------------------------------------------------- |
| `REQUEST_TIMEOUT_MS` | _(unset)_ | Global shortcut — overrides both `FETCH_TIMEOUT_MS` and `STREAM_IDLE_TIMEOUT_MS` defaults. |
| `FETCH_TIMEOUT_MS` | `600000` | Total HTTP request timeout for upstream provider calls. |
| `STREAM_IDLE_TIMEOUT_MS` | `600000` | Max silence between SSE chunks before aborting. Extended-thinking models rarely pause >90s. |
| `FETCH_HEADERS_TIMEOUT_MS` | = `FETCH_TIMEOUT_MS` | Time to receive response headers. |
| `FETCH_BODY_TIMEOUT_MS` | = `FETCH_TIMEOUT_MS` | Time to receive the full response body. |
| `FETCH_CONNECT_TIMEOUT_MS` | `30000` | TCP connection establishment timeout. |
| `FETCH_KEEPALIVE_TIMEOUT_MS` | `4000` | Keep-alive socket idle timeout. |
| `TLS_CLIENT_TIMEOUT_MS` | = `FETCH_TIMEOUT_MS` | TLS fingerprint proxy (wreq-js) timeout. |
| `API_BRIDGE_PROXY_TIMEOUT_MS` | `30000` | Proxy hop timeout for `/v1` bridge requests. |
| `API_BRIDGE_SERVER_REQUEST_TIMEOUT_MS` | `300000` | Overall server request timeout for the bridge. |
| `API_BRIDGE_SERVER_HEADERS_TIMEOUT_MS` | `60000` | Time to send response headers via the bridge. |
| `API_BRIDGE_SERVER_KEEPALIVE_TIMEOUT_MS` | `5000` | Bridge keep-alive idle timeout. |
| `API_BRIDGE_SERVER_SOCKET_TIMEOUT_MS` | `0` | Raw socket timeout (0 = disabled). |
| `SHUTDOWN_TIMEOUT_MS` | `30000` | Grace period on SIGTERM/SIGINT before force-exit. |
| `OMNIROUTE_DEFAULT_FETCH_TIMEOUT_MS` | `120000` | Fallback used by `src/shared/utils/fetchTimeout.ts` when `FETCH_TIMEOUT_MS` is unset. |
| `OMNIROUTE_CHATGPT_TLS_TIMEOUT_MS` | `60000` | Wire-level timeout for the bogdanfinn/tls-client koffi binding (`chatgptTlsClient.ts`). |
| `OMNIROUTE_CHATGPT_TLS_GRACE_MS` | `10000` | JS-side grace added on top of the wire timeout when the native binding is wedged. |
| `OMNIROUTE_CLAUDE_TLS_TIMEOUT_MS` | `60000` | Wire-level timeout for the bogdanfinn/tls-client koffi binding (`claudeTlsClient.ts`). |
| `OMNIROUTE_CLAUDE_TLS_GRACE_MS` | `10000` | JS-side grace added on top of the wire timeout when the native binding is wedged. |
| `OMNIROUTE_PPLX_TLS_TIMEOUT_MS` | `30000` | Wire-level timeout for the bogdanfinn/tls-client koffi binding (`perplexityTlsClient.ts`). |
| `OMNIROUTE_PPLX_TLS_GRACE_MS` | `10000` | JS-side grace added on top of the wire timeout when the native binding is wedged. |
| Variable | Default | Description |
| ---------------------------------------- | -------------------- | ----------------------------------------------------------------------------------------------------------- |
| `REQUEST_TIMEOUT_MS` | _(unset)_ | Global shortcut — overrides both `FETCH_TIMEOUT_MS` and `STREAM_IDLE_TIMEOUT_MS` defaults. |
| `FETCH_TIMEOUT_MS` | `600000` | Total HTTP request timeout for upstream provider calls. |
| `STREAM_IDLE_TIMEOUT_MS` | `600000` | Max silence between SSE chunks before aborting. Extended-thinking models rarely pause >90s. |
| `STREAM_READINESS_TIMEOUT_MS` | `80000` | Time to receive the first non-ping SSE event. Inherits `REQUEST_TIMEOUT_MS` when set. |
| `FETCH_HEADERS_TIMEOUT_MS` | = `FETCH_TIMEOUT_MS` | Time to receive response headers. |
| `FETCH_BODY_TIMEOUT_MS` | = `FETCH_TIMEOUT_MS` | Time to receive the full response body. |
| `FETCH_CONNECT_TIMEOUT_MS` | `30000` | TCP connection establishment timeout. |
| `FETCH_KEEPALIVE_TIMEOUT_MS` | `4000` | Keep-alive socket idle timeout. |
| `TLS_CLIENT_TIMEOUT_MS` | = `FETCH_TIMEOUT_MS` | TLS fingerprint proxy (wreq-js) timeout. |
| `API_BRIDGE_PROXY_TIMEOUT_MS` | `30000` | Proxy hop timeout for `/v1` bridge requests. |
| `API_BRIDGE_SERVER_REQUEST_TIMEOUT_MS` | `300000` | Overall server request timeout for the bridge. |
| `API_BRIDGE_SERVER_HEADERS_TIMEOUT_MS` | `60000` | Time to send response headers via the bridge. |
| `API_BRIDGE_SERVER_KEEPALIVE_TIMEOUT_MS` | `5000` | Bridge keep-alive idle timeout. |
| `API_BRIDGE_SERVER_SOCKET_TIMEOUT_MS` | `0` | Raw socket timeout (0 = disabled). |
| `SHUTDOWN_TIMEOUT_MS` | `30000` | Grace period on SIGTERM/SIGINT before force-exit. |
| `OMNIROUTE_DEFAULT_FETCH_TIMEOUT_MS` | `120000` | Fallback used by `src/shared/utils/fetchTimeout.ts` when `FETCH_TIMEOUT_MS` is unset. |
| `OMNIROUTE_CHATGPT_TLS_TIMEOUT_MS` | `60000` | Wire-level timeout for the bogdanfinn/tls-client koffi binding (`chatgptTlsClient.ts`). |
| `OMNIROUTE_CHATGPT_TLS_GRACE_MS` | `10000` | JS-side grace added on top of the wire timeout when the native binding is wedged. |
| `OMNIROUTE_CLAUDE_TLS_TIMEOUT_MS` | `60000` | Wire-level timeout for the bogdanfinn/tls-client koffi binding (`claudeTlsClient.ts`). |
| `OMNIROUTE_CLAUDE_TLS_GRACE_MS` | `10000` | JS-side grace added on top of the wire timeout when the native binding is wedged. |
| `OMNIROUTE_PPLX_TLS_TIMEOUT_MS` | `30000` | Wire-level timeout for the bogdanfinn/tls-client koffi binding (`perplexityTlsClient.ts`). |
| `OMNIROUTE_PPLX_TLS_GRACE_MS` | `10000` | JS-side grace added on top of the wire timeout when the native binding is wedged. |
| `OMNIROUTE_GROK_TLS_TIMEOUT_MS` | `60000` | Wire-level timeout for the bogdanfinn/tls-client koffi binding (`grokTlsClient.ts`). |
| `OMNIROUTE_GROK_TLS_GRACE_MS` | `10000` | JS-side grace added on top of the wire timeout when the native binding is wedged. |
| `OMNIROUTE_BROWSER_POOL` | `on` | Shared Playwright browser pool for browser-backed web-cookie chat (`browserPool.ts`); set `off` to disable. |
| `WEB_COOKIE_USE_BROWSER` | `0` | Opt a web-cookie chat request into the browser-backed path (`browserBackedChat.ts`); `1` to enable. |
Combo target attempts inherit the resolved upstream request timeout (`FETCH_TIMEOUT_MS`, or
`REQUEST_TIMEOUT_MS` when it supplies the fetch default). Set `targetTimeoutMs` in a combo,
combo defaults, or provider override only to make combo fallback faster; values above the
current upstream timeout are capped to the upstream timeout.
### Circuit Breaker Thresholds
@@ -583,18 +636,18 @@ The logging system writes to both stdout and rotated log files. All configuratio
## 17. Memory Optimization
| Variable | Default | Description |
| -------------------------- | ------------------------------- | ---------------------------------------------------------------------- |
| `OMNIROUTE_MEMORY_MB` | `256` (Docker) / system default | V8 heap limit. Sets `--max-old-space-size`. |
| `PROMPT_CACHE_MAX_SIZE` | `50` | Max cached system prompt entries. |
| `PROMPT_CACHE_MAX_BYTES` | `2097152` (2 MB) | Max total prompt cache size. |
| `PROMPT_CACHE_TTL_MS` | `300000` (5 min) | Prompt cache entry TTL. |
| `SEMANTIC_CACHE_MAX_SIZE` | `100` | Max cached temperature=0 responses. |
| `SEMANTIC_CACHE_MAX_BYTES` | `4194304` (4 MB) | Max total semantic cache size. |
| `SEMANTIC_CACHE_TTL_MS` | `1800000` (30 min) | Semantic cache entry TTL. |
| `STREAM_HISTORY_MAX` | `50` | Max recent stream events in the Dashboard live view buffer. |
| `CONTEXT_LENGTH_DEFAULT` | `128000` | Global fallback max context length for models without explicit config. |
| `USAGE_TOKEN_BUFFER` | `100` | Extra token headroom reserved when tracking usage quotas. |
| Variable | Default | Description |
| -------------------------- | ------------------ | ---------------------------------------------------------------------------------------------------- |
| `OMNIROUTE_MEMORY_MB` | `512` | Runtime V8 heap limit. Docker standalone and `omniroute serve` use it to set `--max-old-space-size`. |
| `PROMPT_CACHE_MAX_SIZE` | `50` | Max cached system prompt entries. |
| `PROMPT_CACHE_MAX_BYTES` | `2097152` (2 MB) | Max total prompt cache size. |
| `PROMPT_CACHE_TTL_MS` | `300000` (5 min) | Prompt cache entry TTL. |
| `SEMANTIC_CACHE_MAX_SIZE` | `100` | Max cached temperature=0 responses. |
| `SEMANTIC_CACHE_MAX_BYTES` | `4194304` (4 MB) | Max total semantic cache size. |
| `SEMANTIC_CACHE_TTL_MS` | `1800000` (30 min) | Semantic cache entry TTL. |
| `STREAM_HISTORY_MAX` | `50` | Max recent stream events in the Dashboard live view buffer. |
| `CONTEXT_LENGTH_DEFAULT` | `128000` | Global fallback max context length for models without explicit config. |
| `USAGE_TOKEN_BUFFER` | `100` | Extra token headroom reserved when tracking usage quotas. |
### Compression
@@ -602,6 +655,21 @@ The logging system writes to both stdout and rotated log files. All configuratio
| ------------------------------------- | ------- | ------------------------------------------------------------------------------------------------------------- |
| `OMNIROUTE_RTK_TRUST_PROJECT_FILTERS` | unset | Trust project `.rtk/filters.json` without a `.rtk/trust.json` hash. Use only in controlled local development. |
### Memory Engine (plan 21)
Embedding layer, vector store and reranking knobs for the persistent memory subsystem (`src/lib/memory/`).
| Variable | Default | Description |
| ------------------------------- | -------------------------- | ---------------------------------------------------------------------------------------------------------- |
| `MEMORY_EMBEDDING_CACHE_TTL_MS` | `300000` (5 min) | TTL for the in-memory embedding cache (per source/model/dim signature). |
| `MEMORY_EMBEDDING_CACHE_MAX` | `1000` | Max LRU entries kept in the embedding cache. |
| `MEMORY_TRANSFORMERS_MODEL` | `Xenova/all-MiniLM-L6-v2` | HF repo id for the opt-in `@huggingface/transformers` local MiniLM pipeline (~23 MB int8, ~400 MB RAM). |
| `MEMORY_STATIC_MODEL` | `minishlab/potion-base-8M` | HF repo id for the static potion/Model2Vec lookup-table embedder. Downloaded lazily into the cache dir. |
| `MEMORY_STATIC_CACHE_DIR` | `<DATA_DIR>/embeddings` | Directory used to cache the static potion model files. Defaults under `DATA_DIR` when unset. |
| `MEMORY_VEC_TOP_K` | `20` | Default top-K used by the `sqlite-vec` brute-force vector search inside `src/lib/memory/vectorStore.ts`. |
| `MEMORY_RRF_K` | `60` | Reciprocal Rank Fusion constant `k` for hybrid FTS5 + vector retrieval (sqlite-vec recipe). |
| `HF_HUB_ENDPOINT` | `https://huggingface.co` | Override Hugging Face Hub base URL used by `staticPotion.ts` (e.g. mirror endpoint for air-gapped setups). |
### Low-RAM Docker Example
```bash
@@ -627,6 +695,15 @@ Automatic model pricing data synchronization from external sources.
---
## Arena ELO Sync
| Variable | Default | Source File | Description |
| ------------------------- | ------------- | ------------------------------------------------ | ------------------------------------------------------------------------------------------------------------- |
| `ARENA_ELO_SYNC_ENABLED` | `true` | `src/shared/constants/featureFlagDefinitions.ts` | Periodic Arena AI leaderboard ELO sync, configurable from Dashboard Feature Flags or with `false` to opt out. |
| `ARENA_ELO_SYNC_INTERVAL` | `86400` (24h) | `src/lib/arenaEloSync.ts` | Sync interval in seconds. |
---
## 19. Model Sync (Dev)
| Variable | Default | Source File | Description |
@@ -637,22 +714,27 @@ Automatic model pricing data synchronization from external sources.
## 20. Provider-Specific Settings
| Variable | Default | Source File | Description |
| ----------------------------------------- | ------------------ | --------------------------------------------------------------------- | ------------------------------------------------------------------------------------- |
| `OPENROUTER_CATALOG_TTL_MS` | `86400000` (24h) | `src/lib/catalog/openrouterCatalog.ts` | OpenRouter model catalog cache TTL. |
| `NANOBANANA_POLL_TIMEOUT_MS` | `120000` | `open-sse/handlers/imageGeneration.ts` | Max wait for NanoBanana image generation jobs. |
| `NANOBANANA_POLL_INTERVAL_MS` | `2500` | `open-sse/handlers/imageGeneration.ts` | NanoBanana job polling frequency. |
| `AWS_REGION` | _(unset)_ | `src/lib/providers/validation.ts`, `open-sse/handlers/audioSpeech.ts` | Region used to construct AWS Bedrock endpoints (Kiro, audio). |
| `AWS_DEFAULT_REGION` | _(unset)_ | `src/lib/providers/validation.ts`, `open-sse/handlers/audioSpeech.ts` | Fallback when `AWS_REGION` is not set. |
| `CLOUDFLARE_ACCOUNT_ID` | _(unset)_ | `open-sse/executors/cloudflare-ai.ts` | Account ID for Cloudflare Workers AI. |
| `CLOUDFLARED_BIN` | auto-detect | `src/lib/cloudflaredTunnel.ts` | Custom path to `cloudflared` binary. |
| `SEARCH_CACHE_TTL_MS` | `300000` (5 min) | `open-sse/services/searchCache.ts` | TTL for search API (Perplexity, Brave, etc.) response caching. |
| `ALLOW_MULTI_CONNECTIONS_PER_COMPAT_NODE` | `false` | `src/app/api/providers/route.ts` | Allow multiple simultaneous connections per OpenAI-compatible provider. |
| `ENABLE_CC_COMPATIBLE_PROVIDER` | `false` | `src/shared/utils/featureFlags.ts` | Reveal the experimental CC-compatible provider UI for Claude Code-only relays. |
| `CLIPROXYAPI_HOST` | `127.0.0.1` | `open-sse/executors/cliproxyapi.ts` | CLIProxyAPI bridge host (legacy integration). |
| `CLIPROXYAPI_PORT` | `5544` | `open-sse/executors/cliproxyapi.ts` | CLIProxyAPI bridge port. |
| `CLIPROXYAPI_CONFIG_DIR` | `~/.cli-proxy-api` | `src/lib/versionManager/processManager.ts` | CLIProxyAPI config directory. |
| `LOCAL_HOSTNAMES` | _(empty)_ | `open-sse/config/providerRegistry.ts` | Comma-separated additional hostnames treated as "local" (Docker service names, etc.). |
| Variable | Default | Source File | Description |
| ----------------------------------------- | ------------------ | --------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
| `OPENROUTER_CATALOG_TTL_MS` | `86400000` (24h) | `src/lib/catalog/openrouterCatalog.ts` | OpenRouter model catalog cache TTL. |
| `MODEL_CATALOG_INCLUDE_NAMES` | `true` | `src/shared/constants/featureFlagDefinitions.ts` | Include display-friendly `name` fields in `/v1/models` responses. Disable for clients that expect IDs only. |
| `NANOBANANA_POLL_TIMEOUT_MS` | `120000` | `open-sse/handlers/imageGeneration.ts` | Max wait for NanoBanana image generation jobs. |
| `NANOBANANA_POLL_INTERVAL_MS` | `2500` | `open-sse/handlers/imageGeneration.ts` | NanoBanana job polling frequency. |
| `AWS_REGION` | _(unset)_ | `src/lib/providers/validation.ts`, `open-sse/handlers/audioSpeech.ts` | Region used to construct AWS Bedrock endpoints (Kiro, audio). |
| `AWS_DEFAULT_REGION` | _(unset)_ | `src/lib/providers/validation.ts`, `open-sse/handlers/audioSpeech.ts` | Fallback when `AWS_REGION` is not set. |
| `CLOUDFLARE_ACCOUNT_ID` | _(unset)_ | `open-sse/executors/cloudflare-ai.ts` | Account ID for Cloudflare Workers AI. |
| `CLOUDFLARED_BIN` | auto-detect | `src/lib/cloudflaredTunnel.ts` | Custom path to `cloudflared` binary. |
| `SEARCH_CACHE_TTL_MS` | `300000` (5 min) | `open-sse/services/searchCache.ts` | TTL for search API (Perplexity, Brave, etc.) response caching. |
| `ALLOW_MULTI_CONNECTIONS_PER_COMPAT_NODE` | `false` | `src/app/api/providers/route.ts` | Allow multiple simultaneous connections per OpenAI-compatible provider. |
| `ENABLE_CC_COMPATIBLE_PROVIDER` | `false` | `src/shared/utils/featureFlags.ts` | Reveal the experimental CC-compatible provider UI for Claude Code-only relays. |
| `NINEROUTER_HOST` | `127.0.0.1` | `open-sse/executors/ninerouter.ts` | Override the host where the embedded 9router instance listens. |
| `NINEROUTER_PORT` | `20130` | `open-sse/executors/ninerouter.ts` | Override the port where the embedded 9router instance listens. |
| `EMBED_WS_PROXY_HOST` | `127.0.0.1` | `src/lib/services/embedWsProxy.ts` | Bind host for the embedded-service WebSocket proxy (loopback only by default). |
| `EMBED_WS_PROXY_PORT` | `20131` | `src/lib/services/embedWsProxy.ts` | Port for the embedded-service WebSocket proxy server. |
| `CLIPROXYAPI_HOST` | `127.0.0.1` | `open-sse/executors/cliproxyapi.ts` | CLIProxyAPI bridge host (legacy integration). |
| `CLIPROXYAPI_PORT` | `5544` | `open-sse/executors/cliproxyapi.ts` | CLIProxyAPI bridge port. |
| `CLIPROXYAPI_CONFIG_DIR` | `~/.cli-proxy-api` | `src/lib/versionManager/processManager.ts` | CLIProxyAPI config directory. |
| `LOCAL_HOSTNAMES` | _(empty)_ | `open-sse/config/providerRegistry.ts` | Comma-separated additional hostnames treated as "local" (Docker service names, etc.). |
`ENABLE_CC_COMPATIBLE_PROVIDER` is only for third-party relays that accept Claude Code clients
exclusively. OmniRoute rewrites requests so those relays accept them. If you only want to use
@@ -663,15 +745,18 @@ Anthropic-compatible provider instead.
## 21. Proxy Health
| Variable | Default | Source File | Description |
| ---------------------------- | ---------------- | ---------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `PROXY_FAST_FAIL_TIMEOUT_MS` | `2000` | `src/lib/proxyHealth.ts` | Fast-fail health check timeout. |
| `PROXY_HEALTH_CACHE_TTL_MS` | `30000` | `src/lib/proxyHealth.ts` | Health check result cache TTL. |
| `RATE_LIMIT_MAX_WAIT_MS` | `120000` (2 min) | `open-sse/services/rateLimitManager.ts` | Max time to wait on a 429 before failing the request. |
| `RATE_LIMIT_AUTO_ENABLE` | _(unset)_ | `open-sse/services/rateLimitManager.ts` | Force the auto-enable rate limit safety net on/off regardless of the persisted Dashboard setting. Accepts `true`/`1`/`on` to force on, `false`/`0`/`off` to force off. |
| `HEALTHCHECK_STAGGER_MS` | `3000` | `src/lib/tokenHealthCheck.ts` | Stagger interval (ms) between provider token healthchecks at startup. |
| `REQUEST_RETRY` | `2` | `src/sse/services/cooldownAwareRetry.ts` | Number of automatic retries on model-scoped cooldown responses before returning error to client. |
| `MAX_RETRY_INTERVAL_SEC` | `30` | `src/sse/services/cooldownAwareRetry.ts` | Max backoff interval (seconds) between cooldown retries. Capped by this value regardless of upstream `Retry-After`. |
| Variable | Default | Source File | Description |
| ---------------------------- | ---------------- | ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `PROXY_FAST_FAIL_TIMEOUT_MS` | `2000` | `src/lib/proxyHealth.ts` | Fast-fail health check timeout. |
| `PROXY_HEALTH_CACHE_TTL_MS` | `30000` | `src/lib/proxyHealth.ts` | Health check result cache TTL. |
| `RATE_LIMIT_MAX_WAIT_MS` | `120000` (2 min) | `open-sse/services/rateLimitManager.ts` | Max time to wait on a 429 before failing the request. |
| `RATE_LIMIT_AUTO_ENABLE` | _(unset)_ | `open-sse/services/rateLimitManager.ts` | Force the auto-enable rate limit safety net on/off regardless of the persisted Dashboard setting. Accepts `true`/`1`/`on` to force on, `false`/`0`/`off` to force off. |
| `PROVIDER_COOLDOWN_ENABLED` | _(unset → off)_ | `open-sse/services/providerCooldownTracker.ts` | Opt-in global cross-request provider/connection cooldown tracking. OFF by default (overlaps Connection Cooldown / Provider Circuit Breaker). Accepts `true`/`1`/`on` to enable. |
| `PROVIDER_COOLDOWN_MIN_MS` | `5000` | `open-sse/services/providerCooldownTracker.ts` | Minimum cooldown (ms) before a failed provider/connection is retried. Scaled exponentially with consecutive failures. Only used when `PROVIDER_COOLDOWN_ENABLED`. |
| `PROVIDER_COOLDOWN_MAX_MS` | `300000` (5 min) | `open-sse/services/providerCooldownTracker.ts` | Maximum cooldown (ms) cap before a failed provider/connection is retried regardless. Only used when `PROVIDER_COOLDOWN_ENABLED`. |
| `HEALTHCHECK_STAGGER_MS` | `3000` | `src/lib/tokenHealthCheck.ts` | Stagger interval (ms) between provider token healthchecks at startup. |
| `REQUEST_RETRY` | `2` | `src/sse/services/cooldownAwareRetry.ts` | Number of automatic retries on model-scoped cooldown responses before returning error to client. |
| `MAX_RETRY_INTERVAL_SEC` | `30` | `src/sse/services/cooldownAwareRetry.ts` | Max backoff interval (seconds) between cooldown retries. Capped by this value regardless of upstream `Retry-After`. |
---
@@ -680,17 +765,19 @@ Anthropic-compatible provider instead.
> [!CAUTION]
> These variables produce **verbose output** and may leak sensitive data. **Never enable in production.**
| Variable | Default | Source File | Description |
| -------------------------------- | ------------------- | ------------------------------------------ | --------------------------------------------------------------------------------- |
| `CURSOR_DEBUG` | _(unset)_ | `open-sse/executors/cursor.ts` | Set `1` to enable verbose Cursor executor logs (decoded SSE chunks, etc.). |
| `CURSOR_STREAM_DEBUG` | _(unset)_ | `open-sse/executors/cursor.ts` | Backward-compatible alias of `CURSOR_DEBUG`. |
| `CURSOR_DUMP_FILE` | _(unset)_ | `open-sse/executors/cursor.ts` | Optional file path that receives raw decoded Cursor chunks when `CURSOR_DEBUG=1`. |
| `CURSOR_STREAM_TIMEOUT_MS` | `300000` | `open-sse/executors/cursor.ts` | Stream idle timeout (ms) for the Cursor executor. |
| `CURSOR_STATE_DB_PATH` | _(probed)_ | `open-sse/utils/cursorVersionDetector.ts` | Override the Cursor state DB lookup used for version detection. |
| `CURSOR_TOKEN` | _(unset)_ | `scripts/ad-hoc/cursor-tap.cjs` | Direct Cursor bearer token used by developer tooling. |
| `OMNIROUTE_LOG_REQUEST_SHAPE` | enabled (`!== "0"`) | `src/app/api/v1/chat/completions/route.ts` | Log content-type/length markers for large chat payloads. Set `"0"` to silence. |
| `DEBUG_RESPONSES_SSE_TO_JSON` | _(unset)_ | `open-sse/handlers/responseTranslator.ts` | Set `true` to log Responses API SSE→JSON translation details. |
| `NEXT_PUBLIC_OMNIROUTE_E2E_MODE` | _(unset)_ | E2E test harness | Set `true` to enable E2E test mode (relaxed auth, test hooks). |
| Variable | Default | Source File | Description |
| -------------------------------- | ------------------- | ------------------------------------------ | -------------------------------------------------------------------------------------------- |
| `CURSOR_DEBUG` | _(unset)_ | `open-sse/executors/cursor.ts` | Set `1` to enable verbose Cursor executor logs (decoded SSE chunks, etc.). |
| `CURSOR_STREAM_DEBUG` | _(unset)_ | `open-sse/executors/cursor.ts` | Backward-compatible alias of `CURSOR_DEBUG`. |
| `CURSOR_DUMP_FILE` | _(unset)_ | `open-sse/executors/cursor.ts` | Optional file path that receives raw decoded Cursor chunks when `CURSOR_DEBUG=1`. |
| `CURSOR_STREAM_TIMEOUT_MS` | `300000` | `open-sse/executors/cursor.ts` | Stream idle timeout (ms) for the Cursor executor. |
| `CURSOR_TOOL_DIRECTIVE` | enabled (`!== "0"`) | `open-sse/executors/cursor.ts` | Tool-commit directive that makes composer-2.5 reliably issue tool calls. Set `0` to disable. |
| `CURSOR_IMAGE_FETCH_TIMEOUT_MS` | `15000` | `open-sse/utils/cursorImages.ts` | Per-image fetch timeout (ms) for remote `image_url` vision input. |
| `CURSOR_STATE_DB_PATH` | _(probed)_ | `open-sse/utils/cursorVersionDetector.ts` | Override the Cursor state DB lookup used for version detection. |
| `CURSOR_TOKEN` | _(unset)_ | `scripts/ad-hoc/cursor-tap.cjs` | Direct Cursor bearer token used by developer tooling. |
| `OMNIROUTE_LOG_REQUEST_SHAPE` | enabled (`!== "0"`) | `src/app/api/v1/chat/completions/route.ts` | Log content-type/length markers for large chat payloads. Set `"0"` to silence. |
| `DEBUG_RESPONSES_SSE_TO_JSON` | _(unset)_ | `open-sse/handlers/responseTranslator.ts` | Set `true` to log Responses API SSE→JSON translation details. |
| `NEXT_PUBLIC_OMNIROUTE_E2E_MODE` | _(unset)_ | E2E test harness | Set `true` to enable E2E test mode (relaxed auth, test hooks). |
---
@@ -790,26 +877,58 @@ Limits and safety knobs applied when the Skills framework (`src/lib/skills/`) ex
Provider quota endpoints, network tunnels (Tailscale, Ngrok, MITM debug proxy), the 1Proxy egress pool, database backups and small per-feature overrides referenced by the executor layer or scripts.
| Variable | Default | Source File | Description |
| -------------------------------- | ------------------------------------- | --------------------------------------------------- | --------------------------------------------------------------------------- |
| `REDIS_URL` | `redis://localhost:6379` | `src/shared/utils/rateLimiter.ts` | Redis connection string for the rate limiter backend. |
| `ALIBABA_CODING_PLAN_HOST` | _(production host)_ | `open-sse/services/bailianQuotaFetcher.ts` | Override the host used to fetch Alibaba Bailian coding-plan quotas. |
| `ALIBABA_CODING_PLAN_QUOTA_URL` | derived from host | `open-sse/services/bailianQuotaFetcher.ts` | Full quota URL override for Alibaba Bailian. |
| `CONTEXT_RESERVE_TOKENS` | `1024` | `open-sse/services/contextManager.ts` | Tokens reserved for completion output when computing prompt budgets. |
| `MODEL_ALIAS_COMPAT_ENABLED` | enabled | `open-sse/services/model.ts` | Toggle the legacy model-alias compatibility layer used by older clients. |
| `COMMAND_CODE_CALLBACK_PORT` | _(unset)_ | `src/app/api/providers/command-code/auth/shared.ts` | Local port used for OAuth-style callbacks from the Command Code CLI helper. |
| `MITM_LOCAL_PORT` | `443` | `src/mitm/server.cjs` | Local bind port for the MITM debug proxy. |
| `MITM_DISABLE_TLS_VERIFY` | `0` | `src/mitm/server.cjs` | Set `1` to disable upstream TLS verification (development only). |
| `ONEPROXY_ENABLED` | `true` | `src/lib/oneproxySync.ts` | Enable the 1Proxy egress pool sync. |
| `ONEPROXY_API_URL` | `https://1proxy-api.aitradepulse.com` | `src/lib/oneproxySync.ts` | 1Proxy service API URL override. |
| `ONEPROXY_MAX_PROXIES` | `500` | `src/lib/oneproxySync.ts` | Maximum proxies imported per sync. |
| `ONEPROXY_MIN_QUALITY_THRESHOLD` | `50` | `src/lib/oneproxySync.ts` | Minimum quality score for imported proxies. |
| `TAILSCALE_BIN` | _(auto-detect)_ | `src/lib/tailscaleTunnel.ts` | Explicit path to the `tailscale` binary. |
| `TAILSCALED_BIN` | _(auto-detect)_ | `src/lib/tailscaleTunnel.ts` | Explicit path to the `tailscaled` daemon binary. |
| `NGROK_AUTHTOKEN` | _(unset)_ | `src/lib/ngrokTunnel.ts` | Authenticates outbound ngrok tunnels. |
| `DB_BACKUP_MAX_FILES` | `20` | `src/lib/db/backup.ts` | Maximum SQLite backup files retained on disk. |
| `DB_BACKUP_RETENTION_DAYS` | `0` | `src/lib/db/backup.ts` | Maximum age (days) of retained backups. `0` disables age-based pruning. |
| `OMNIROUTE_TLS_PROXY_URL` | _(unset)_ | `open-sse/services/chatgptTlsClient.ts` | Override the TLS sidecar URL for tests. Production should leave unset. |
| Variable | Default | Source File | Description |
| ------------------------------------------ | --------------------------------------------------------------------------- | ------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `REDIS_URL` | `redis://localhost:6379` | `src/shared/utils/rateLimiter.ts` | Redis connection string for the rate limiter backend. |
| `ALIBABA_CODING_PLAN_HOST` | _(production host)_ | `open-sse/services/bailianQuotaFetcher.ts` | Override the host used to fetch Alibaba Bailian coding-plan quotas. |
| `ALIBABA_CODING_PLAN_QUOTA_URL` | derived from host | `open-sse/services/bailianQuotaFetcher.ts` | Full quota URL override for Alibaba Bailian. |
| `CONTEXT_RESERVE_TOKENS` | `1024` | `open-sse/services/contextManager.ts` | Tokens reserved for completion output when computing prompt budgets. |
| `MODEL_ALIAS_COMPAT_ENABLED` | enabled | `open-sse/services/model.ts` | Toggle the legacy model-alias compatibility layer used by older clients. |
| `OMNIROUTE_EMERGENCY_FALLBACK` | enabled | `open-sse/services/emergencyFallback.ts` | Set `false` (or `0`) to disable the emergency budget-exhaustion fallback that reroutes failed requests to the free `nvidia`/`openai/gpt-oss-120b` model. Effective precedence is Feature Flags DB override > env var > default; if unavailable, the service falls back to the raw env value. |
| `COMMAND_CODE_CALLBACK_PORT` | _(unset)_ | `src/app/api/providers/command-code/auth/shared.ts` | Local port used for OAuth-style callbacks from the Command Code CLI helper. |
| `COMMAND_CODE_VERSION` | `0.33.2` | `open-sse/executors/commandCode.ts` | Value sent as the `x-command-code-version` header to the Command Code upstream. Override to bump the CLI version. |
| `MITM_LOCAL_PORT` | `443` | `src/mitm/server.cjs` | Local bind port for the MITM debug proxy. |
| `MITM_DISABLE_TLS_VERIFY` | `0` | `src/mitm/server.cjs` | Set `1` to disable upstream TLS verification (development only). |
| `ONEPROXY_ENABLED` | `true` | `src/lib/oneproxySync.ts` | Enable the 1Proxy egress pool sync. |
| `ONEPROXY_API_URL` | `https://1proxy-api.aitradepulse.com` | `src/lib/oneproxySync.ts` | 1Proxy service API URL override. |
| `ONEPROXY_MAX_PROXIES` | `500` | `src/lib/oneproxySync.ts` | Maximum proxies imported per sync. |
| `ONEPROXY_MIN_QUALITY_THRESHOLD` | `50` | `src/lib/oneproxySync.ts` | Minimum quality score for imported proxies. |
| `FREE_PROXY_1PROXY_ENABLED` | `true` | `src/lib/freeProxyProviders/oneproxy.ts` | Enable the 1proxy free proxy source. Set to `false` to disable. |
| `FREE_PROXY_1PROXY_API_URL` | _(see oneproxy.ts)_ | `src/lib/freeProxyProviders/oneproxy.ts` | 1proxy API URL override. |
| `FREE_PROXY_1PROXY_MAX` | `500` | `src/lib/freeProxyProviders/oneproxy.ts` | Maximum proxies fetched per sync from 1proxy. |
| `FREE_PROXY_1PROXY_MIN_QUALITY` | `50` | `src/lib/freeProxyProviders/oneproxy.ts` | Minimum quality score threshold for 1proxy imports. |
| `FREE_PROXY_PROXIFLY_ENABLED` | `true` | `src/lib/freeProxyProviders/proxifly.ts` | Enable the Proxifly free proxy source. Set to `false` to disable. |
| `FREE_PROXY_PROXIFLY_QUANTITY` | `100` | `src/lib/freeProxyProviders/proxifly.ts` | Number of proxies to fetch per Proxifly sync. |
| `FREE_PROXY_PROXIFLY_ANONYMITY` | `elite` | `src/lib/freeProxyProviders/proxifly.ts` | Anonymity level filter for Proxifly (`elite`, `anonymous`, `transparent`). |
| `FREE_PROXY_IPLOCATE_ENABLED` | `false` | `src/lib/freeProxyProviders/iplocate.ts` | Enable the IPLocate free proxy source. Opt-in only. |
| `FREE_PROXY_IPLOCATE_BASE_URL` | `https://raw.githubusercontent.com/iplocate/free-proxy-list/main/protocols` | `src/lib/freeProxyProviders/iplocate.ts` | IPLocate proxy list base URL override. |
| `NEXT_PUBLIC_VERCEL_RELAY_ENABLED` | `true` | `src/app/(dashboard)/…/ProxyPoolTab.tsx` | Show/hide the Deploy Vercel Relay button in the Proxy Pool tab. |
| `VERCEL_API_BASE` | `https://api.vercel.com` | `src/app/api/settings/proxy/vercel-deploy/route.ts` | Vercel API base URL override (for testing). |
| `NEXT_PUBLIC_VERCEL_RELAY_DEFAULT_PROJECT` | `omniroute-relay` | `src/app/(dashboard)/…/VercelRelayModal.tsx` | Default project name pre-filled in the Vercel Relay deploy modal. |
| `TAILSCALE_BIN` | _(auto-detect)_ | `src/lib/tailscaleTunnel.ts` | Explicit path to the `tailscale` binary. |
| `TAILSCALED_BIN` | _(auto-detect)_ | `src/lib/tailscaleTunnel.ts` | Explicit path to the `tailscaled` daemon binary. |
| `NGROK_AUTHTOKEN` | _(unset)_ | `src/lib/ngrokTunnel.ts` | Authenticates outbound ngrok tunnels. |
| `DB_BACKUP_MAX_FILES` | `20` | `src/lib/db/backup.ts` | Maximum SQLite backup files retained on disk. |
| `DB_BACKUP_RETENTION_DAYS` | `0` | `src/lib/db/backup.ts` | Maximum age (days) of retained backups. `0` disables age-based pruning. |
| `OMNIROUTE_TLS_PROXY_URL` | _(unset)_ | `open-sse/services/chatgptTlsClient.ts` | Override the TLS sidecar URL for tests. Production should leave unset. |
| `CONTAINER_HOST` | `docker` | `scripts/check-permissions.sh` | Container runtime hint for the entrypoint permission check. Set to `podman` under rootless Podman so the fix instructions use `podman unshare chown` instead of `sudo chown`. |
| `QUOTA_STORE_DRIVER` | `sqlite` | `src/lib/quota/storeFactory.ts` | Quota-share consumption store backend: `sqlite` (default) or `redis`. |
| `QUOTA_STORE_REDIS_URL` | _(unset)_ | `src/lib/quota/storeFactory.ts` | Redis connection string used when `QUOTA_STORE_DRIVER=redis` (e.g. `redis://localhost:6379`). |
| `QUOTA_SATURATION_THRESHOLD` | `0.5` | `src/lib/quota/enforce.ts` | Pool saturation ratio (0..1); at/above it the pool enters strict mode (no borrowing). |
| `QUOTA_SOFT_DEPRIORITIZE_FACTOR` | `0.7` | `open-sse/services/combo.ts` | Score multiplier (0..1) applied to a target when the soft quota policy deprioritizes it. |
| `QUOTA_CONSUMPTION_RETENTION_DAYS` | `14` | `src/lib/db/quotaConsumption.ts` | Retention window (days) for `quota_consumption` buckets before GC (`gcQuotaConsumption`). |
| `AGENTBRIDGE_UPSTREAM_CA_CERT` | _(unset)_ | `src/mitm/manager.ts` | Extra CA certificate (PEM) trusted for AgentBridge upstream TLS connections. |
| `INSPECTOR_BUFFER_SIZE` | `1000` | `src/mitm/inspector/buffer.ts` | Max captured requests held in the Traffic Inspector ring buffer. |
| `INSPECTOR_MAX_BODY_KB` | `1024` | `src/mitm/inspector/buffer.ts` | Max captured request/response body size (KB) before truncation. |
| `INSPECTOR_HTTP_PROXY_PORT` | `8080` | `src/mitm/inspector/httpProxyServer.ts` | Local port for the Traffic Inspector HTTP proxy. |
| `INSPECTOR_HTTP_PROXY_AUTOSTART` | `false` | `src/mitm/inspector/httpProxyServer.ts` | Auto-start the inspector HTTP proxy on boot. |
| `INSPECTOR_TLS_INTERCEPT` | `false` | `src/lib/inspector/captureState.ts` | Enable TLS interception (MITM) for captured HTTPS traffic. |
| `INSPECTOR_LLM_HOSTS_EXTRA` | _(unset)_ | `src/lib/inspector/captureState.ts` | Extra hostnames (comma-separated) treated as LLM endpoints for capture. |
| `INSPECTOR_MASK_SECRETS` | `true` | `src/mitm/inspector/buffer.ts` | Mask secrets (auth headers / API keys) in captured traffic. |
| `INSPECTOR_SYSTEM_PROXY_GUARD_MINUTES` | `30` | `src/app/api/tools/traffic-inspector/capture-modes/system-proxy/route.ts` | Minutes before the system-proxy guard auto-reverts OS proxy settings. |
| `INSPECTOR_INTERNAL_INGEST_TOKEN` | _(auto)_ | `src/app/api/tools/traffic-inspector/internal/ingest/route.ts` | Token authenticating internal capture ingest into the inspector. |
| `PLAYGROUND_COMPARE_MAX_COLUMNS` | `4` | `src/app/(dashboard)/dashboard/playground/` | Max number of side-by-side columns in the Playground compare mode. |
| `PLAYGROUND_IMPROVE_PROMPT_DEFAULT_MODEL` | _(unset)_ | `src/app/(dashboard)/dashboard/playground/` | Default model for the Playground 'improve prompt' action (falls back to the active model when unset). |
---
@@ -819,24 +938,25 @@ Used by `scripts/dev/run-next-playwright.mjs`, `scripts/dev/smoke-electron-packa
`scripts/dev/run-ecosystem-tests.mjs`, and `scripts/build/uninstall.mjs`. Leave every
value below unset in production deployments.
| Variable | Default | Source File | Description |
| ------------------------------------- | -------------------------------- | ----------------------------------------- | ---------------------------------------------------------------------------------------- |
| `OMNIROUTE_E2E_BOOTSTRAP_MODE` | `auth` | `scripts/dev/run-next-playwright.mjs` | E2E bootstrap mode (`auth`, `fresh`, `reuse`) for the Playwright runner. |
| `OMNIROUTE_E2E_PASSWORD` | falls back to `INITIAL_PASSWORD` | `scripts/dev/run-next-playwright.mjs` | Admin password injected into the Playwright environment. |
| `OMNIROUTE_DISABLE_LOCAL_HEALTHCHECK` | `true` | `scripts/dev/run-next-playwright.mjs` | Disable the local healthcheck poll during Playwright runs. |
| `OMNIROUTE_DISABLE_TOKEN_HEALTHCHECK` | `true` | `scripts/dev/run-next-playwright.mjs` | Disable the OAuth token healthcheck loop during tests. |
| `OMNIROUTE_HIDE_HEALTHCHECK_LOGS` | `true` | `scripts/dev/run-next-playwright.mjs` | Silence healthcheck noise in Playwright stdout. |
| `OMNIROUTE_PLAYWRIGHT_SKIP_BUILD` | `0` | `scripts/dev/run-next-playwright.mjs` | Skip the Next.js production build before Playwright starts (CI optimization). |
| `OMNIROUTE_SKIP_UNINSTALL_HOOK` | `0` | `scripts/build/uninstall.mjs` | Skip the OmniRoute uninstall hook (used by CI to keep `node_modules` intact). |
| `ECOSYSTEM_SERVER_WAIT_MS` | `180000` | `scripts/dev/run-ecosystem-tests.mjs` | Wait time (ms) for the server to become healthy before running ecosystem/protocol tests. |
| `ELECTRON_SMOKE_URL` | `http://127.0.0.1:20128/login` | `scripts/dev/smoke-electron-packaged.mjs` | URL the Electron smoke harness expects the packaged app to serve. |
| `ELECTRON_SMOKE_TIMEOUT_MS` | `45000` | `scripts/dev/smoke-electron-packaged.mjs` | Total timeout (ms) before the smoke harness gives up. |
| `ELECTRON_SMOKE_SETTLE_MS` | `2000` | `scripts/dev/smoke-electron-packaged.mjs` | Settle window (ms) after the page loads. |
| `ELECTRON_SMOKE_APP_EXECUTABLE` | _(auto)_ | `scripts/dev/smoke-electron-packaged.mjs` | Explicit path to the packaged Electron executable. |
| `ELECTRON_SMOKE_DATA_DIR` | _(tmpdir)_ | `scripts/dev/smoke-electron-packaged.mjs` | Data directory for the Electron smoke run. |
| `ELECTRON_SMOKE_KEEP_DATA` | `0` | `scripts/dev/smoke-electron-packaged.mjs` | Set `1` to preserve the smoke data directory after the run. |
| `ELECTRON_SMOKE_STREAM_LOGS` | `0` | `scripts/dev/smoke-electron-packaged.mjs` | Set `1` to stream Electron logs to stdout during the run. |
| `CLI_DEVIN_BIN` | _(PATH lookup)_ | `open-sse/executors/devin-cli.ts` | Override the Devin CLI binary path. |
| Variable | Default | Source File | Description |
| -------------------------------------- | -------------------------------- | ----------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `OMNIROUTE_E2E_BOOTSTRAP_MODE` | `auth` | `scripts/dev/run-next-playwright.mjs` | E2E bootstrap mode (`auth`, `fresh`, `reuse`) for the Playwright runner. |
| `OMNIROUTE_E2E_PASSWORD` | falls back to `INITIAL_PASSWORD` | `scripts/dev/run-next-playwright.mjs` | Admin password injected into the Playwright environment. |
| `OMNIROUTE_DISABLE_LOCAL_HEALTHCHECK` | `true` | `scripts/dev/run-next-playwright.mjs` | Disable the local healthcheck poll during Playwright runs. |
| `OMNIROUTE_DISABLE_TOKEN_HEALTHCHECK` | `true` | `scripts/dev/run-next-playwright.mjs` | Disable the OAuth token healthcheck loop during tests. |
| `OMNIROUTE_HEALTHCHECK_SKIP_PROVIDERS` | _(unset)_ | `src/lib/tokenHealthCheck.ts` | Comma-separated providers excluded from the proactive token-refresh sweep (e.g. `codex,openai`). Targeted alternative to fully disabling the healthcheck — short-TTL providers keep refreshing while cascade providers stay reactive-only. |
| `OMNIROUTE_HIDE_HEALTHCHECK_LOGS` | `true` | `scripts/dev/run-next-playwright.mjs` | Silence healthcheck noise in Playwright stdout. |
| `OMNIROUTE_PLAYWRIGHT_SKIP_BUILD` | `0` | `scripts/dev/run-next-playwright.mjs` | Skip the Next.js production build before Playwright starts (CI optimization). |
| `OMNIROUTE_SKIP_UNINSTALL_HOOK` | `0` | `scripts/build/uninstall.mjs` | Skip the OmniRoute uninstall hook (used by CI to keep `node_modules` intact). |
| `ECOSYSTEM_SERVER_WAIT_MS` | `180000` | `scripts/dev/run-ecosystem-tests.mjs` | Wait time (ms) for the server to become healthy before running ecosystem/protocol tests. |
| `ELECTRON_SMOKE_URL` | `http://127.0.0.1:20128/login` | `scripts/dev/smoke-electron-packaged.mjs` | URL the Electron smoke harness expects the packaged app to serve. |
| `ELECTRON_SMOKE_TIMEOUT_MS` | `45000` | `scripts/dev/smoke-electron-packaged.mjs` | Total timeout (ms) before the smoke harness gives up. |
| `ELECTRON_SMOKE_SETTLE_MS` | `2000` | `scripts/dev/smoke-electron-packaged.mjs` | Settle window (ms) after the page loads. |
| `ELECTRON_SMOKE_APP_EXECUTABLE` | _(auto)_ | `scripts/dev/smoke-electron-packaged.mjs` | Explicit path to the packaged Electron executable. |
| `ELECTRON_SMOKE_DATA_DIR` | _(tmpdir)_ | `scripts/dev/smoke-electron-packaged.mjs` | Data directory for the Electron smoke run. |
| `ELECTRON_SMOKE_KEEP_DATA` | `0` | `scripts/dev/smoke-electron-packaged.mjs` | Set `1` to preserve the smoke data directory after the run. |
| `ELECTRON_SMOKE_STREAM_LOGS` | `0` | `scripts/dev/smoke-electron-packaged.mjs` | Set `1` to stream Electron logs to stdout during the run. |
| `CLI_DEVIN_BIN` | _(PATH lookup)_ | `open-sse/executors/devin-cli.ts` | Override the Devin CLI binary path. |
### Docs translation pipeline
@@ -878,3 +998,16 @@ The following variables appeared in previous versions of `.env.example` but have
| ------------------------- | ------------------------ | ------------------- | ------------------------------------------------------ |
| `APP_LOG_RETENTION_DAYS` | `90` | `7` | ✅ Removed misleading value; documented `7` as default |
| `CALL_LOG_RETENTION_DAYS` | `90` | `7` | ✅ Removed misleading value; documented `7` as default |
### OpenCode config regeneration (ad-hoc tooling)
Used by `scripts/ad-hoc/regen-opencode-config.ts` to regenerate an `opencode.json`
with accurate `limit.context` and `limit.output` values pulled from the running
OmniRoute instance. None of these are required for normal operation — the script
is developer tooling only.
| Variable | Default | Source File | Description |
| ------------------ | ------------------------ | ----------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| `OMNIROUTE_URL` | `http://localhost:20128` | `scripts/ad-hoc/regen-opencode-config.ts` | Base URL of the OmniRoute instance to query for `/v1/models`. |
| `OMNIROUTE_KEY` | _(unset)_ | `scripts/ad-hoc/regen-opencode-config.ts` | API key to authenticate against the OmniRoute `/v1/models` endpoint. Falls back to `OPENCODE_API_KEY` when unset. |
| `OPENCODE_API_KEY` | _(unset)_ | `scripts/ad-hoc/regen-opencode-config.ts` | OpenCode-style API key (`sk-...`) written into the regenerated `opencode.json`. Falls back to `OMNIROUTE_KEY` when unset. |

6
Features.md

@@ -50,13 +50,15 @@ The v3.7.x → v3.8.0 cycle added zero-config auto routing, new providers, OAuth
Manage AI provider connections: OAuth providers (Claude Code, Codex, Gemini CLI), API key providers (Groq, DeepSeek, OpenRouter), and free providers (Qoder, Qwen, Kiro). Kiro accounts include credit balance tracking — remaining credits, total allowance, and renewal date visible in Dashboard → Usage.
OpenRouter connections can store a per-connection `preset` in Advanced Settings. When set, OmniRoute sends it as the OpenRouter top-level request field, for example `"preset": "email-copywriter"`, unless the client request already supplied its own `preset`.
![Providers Dashboard](../screenshots/01-providers.png)
---
## 🎨 Combos
Create model routing combos with 15 strategies: priority, weighted, fill-first, round-robin, p2c (power-of-two-choices), random, least-used, cost-optimized, reset-aware, strict-random, auto, lkgp (last-known-good-provider), context-optimized, and **context-relay**. Each combo chains multiple models with automatic fallback and includes quick templates and readiness checks.
Create model routing combos with 14 strategies: priority, weighted, fill-first, round-robin, p2c (power-of-two-choices), random, least-used, cost-optimized, reset-aware, strict-random, auto, lkgp (last-known-good-provider), context-optimized, and **context-relay**. Each combo chains multiple models with automatic fallback and includes quick templates and readiness checks.
Recent combo improvements:
@@ -182,7 +184,7 @@ Comprehensive proxy configuration enforcement across the entire request pipeline
## 📧 Email Privacy Masking _(v3.5.6+)_
OAuth account emails are now masked in the provider dashboard (e.g. `di*****@g****.com`) to prevent accidental exposure when sharing screenshots or recording demos. The full email address remains accessible via hover tooltip (`title` attribute).
OAuth account emails are masked by default (e.g. `di*****@g****.com`) to prevent accidental exposure when sharing screenshots or recording demos. Use Settings → Appearance → Account email visibility to reveal or mask full account emails globally across providers, combos, logs, quota, and playground screens.
---

10
Fly-io-Deployment-Guide.md

@@ -203,16 +203,14 @@ flyctl deploy
2. **在 provider 控制台配置回调 URL (configure the callback URL on the provider console / configure a URL de callback no painel do provider)**
通常格式为 (typical format / formato típico)：
所有 OAuth provider 共用同一个回调路径 `/callback`，不带 provider 段 (all OAuth providers share the single callback path `/callback` — there is NO per-provider callback route / todos os providers OAuth usam o mesmo callback `/callback`, sem segmento por provider)：
```text
<NEXT_PUBLIC_BASE_URL>/api/oauth/<provider>/callback
<NEXT_PUBLIC_BASE_URL>/callback
```
例如 (e.g. / p.ex.)：
- `https://omniroute.fly.dev/api/oauth/gemini/callback`
- `https://omniroute.fly.dev/api/oauth/antigravity/callback`
- `https://omniroute.fly.dev/api/oauth/cursor/callback`
例如 (e.g. / p.ex.)，无论是 Gemini、Antigravity、Cursor 还是 GitLab Duo (regardless of Gemini / Antigravity / Cursor / GitLab Duo, etc.)：
- `https://omniroute.fly.dev/callback`
如果 `NEXT_PUBLIC_BASE_URL` 与 provider 控制台中注册的回调 URL 不一致，OAuth 流程会在浏览器回跳阶段失败 (mismatch between `NEXT_PUBLIC_BASE_URL` and the registered callback URL will cause OAuth to fail at the browser redirect step / divergência entre `NEXT_PUBLIC_BASE_URL` e a URL de callback registrada quebra o OAuth no redirect do navegador)。

530
Free-Tiers.md

@@ -1,236 +1,356 @@
> 🌍 [View in other languages](Languages)
# Free Tiers
# Free Tiers & Free-Token Budget
> **Last consolidated:** 2026-05-13 — OmniRoute v3.8.2
> **Source of truth:** `src/shared/constants/providers.ts` (`FREE_PROVIDERS`, `OAUTH_PROVIDERS`, and `APIKEY_PROVIDERS` entries flagged with `hasFree: true` + `freeNote`)
> **For Users**: Looking for a simple guide? See the [Free Tiers Guide](../getting-started/FREE-TIERS-GUIDE.md) for step-by-step instructions on getting free AI.
This page lists providers with usable free tiers shipped in OmniRoute v3.8.2. The data is derived from the provider catalog. If a provider does not appear here, it either has no free tier in the catalog or its `hasFree` flag is `false`.
> **Last researched:** 2026-06-05 — per-provider web research of current free-tier quotas + ToS (98 providers).
> **Source of truth (catalog):** `src/shared/constants/providers.ts` (`hasFree: true` + `freeNote`). The token-budget numbers below come from live web research and are an **approximation** — see [Methodology & caveats](#methodology--caveats).
Add credentials from the dashboard (`/dashboard/providers/new`) — OmniRoute reads keys from the database, not from per-provider environment variables. The only env vars that influence provider behavior are listed in the [Environment Variables](#environment-variables) section.
## TL;DR — how much free inference does OmniRoute actually aggregate?
| Metric | Tokens / month | Meaning |
|---|---|---|
| **Documented recurring grant (steady)** | **~1.94B** | 50 provider free-tier **pools** (per-model catalog), each shared pool counted **once**. The live source behind `/api/free-tier/summary` and the dashboard's Free-Tier Budget page. **Use this number.** |
| **+ first month with signup credits** | **~2.53B** | Steady + one-time signup credits (DeepSeek 5M, Together, Jina, …), deduped per account. **First month only** — does not recur. |
| Theoretical ceiling (all rate limits, 24/7) | ~10.87B | Sum of every provider rate limit extrapolated to non-stop use. **Not a guarantee** — do not headline this. |
**Honest headline:** *OmniRoute aggregates **over 1.9B documented free tokens per month** (up to ~2.5B in your first month with signup credits) across 50+ free-tier pools — and RTK + Caveman compression (15–95% token savings) stretches that further.*
> The earlier **~1.54B** figure was a conservative per-PROVIDER estimate (22 hand-picked providers). The **~1.94B** above is the per-MODEL catalog (530 models / 50 pools, `open-sse/config/freeModelCatalog.ts`) — now the canonical source. Both use pool deduplication; the per-model catalog is simply more complete.
Biggest **documented** contributors: `mistral` 1.00B, `longcat` 150M, `cloudflare-ai` 122M, `gemini` 60M, `doubao` 60M, `cerebras` 30M.
> ⚠️ The theoretical ceiling (~10.87B) is inflated by rate-limit-only providers with **no published token cap** (`tencent`, `siliconflow`, `nvidia`, `baidu`, `publicai`, `sparkdesk`) whose figures are `RPM/TPM × 24/7 × 30d` — a theoretical maximum no single account will sustain. They are **excluded** from the defensible number. This is the same inflation that makes competitors' multi-billion claims unreliable.
---
## How free providers are wired
## Methodology & caveats
OmniRoute classifies providers into the following groups in `src/shared/constants/providers.ts`:
| Group | Auth | Example IDs |
| ------------------ | ------------------------------ | ------------------------------------------- |
| `FREE_PROVIDERS` | OAuth or vendor account | `qoder`, `gemini-cli`, `kiro`, `amazon-q` |
| `OAUTH_PROVIDERS` | OAuth | `claude`, `cursor`, `windsurf`, `devin-cli` |
| `APIKEY_PROVIDERS` | API key (with `hasFree: true`) | `groq`, `cerebras`, `mistral`, `gemini` |
A provider appears in the **free pool** when either:
- It is listed in the `FREE_PROVIDERS` map (and not flagged `deprecated: true`), or
- It is listed in `APIKEY_PROVIDERS` with `hasFree: true` and a `freeNote` string, or
- It is an OAuth provider whose vendor offers a free tier on top of OAuth sign-in.
- Numbers are **upper-bound estimates** from each provider's documented free-tier limits as of **2026-06-05**, gathered by web research (confidence tagged per row). Free tiers change constantly — re-verify before relying on a figure.
- `estMonthlyFreeTokens` = recurring monthly tokens only. **One-time signup credits do not recur** and count as 0 (29 providers are signup-credit-only). Discontinued tiers (6) are also 0.
- Daily token cap → `monthly = daily × 30`. Only RPD documented → `RPD × ~800 output tokens × 30`. Only RPM/TPM (no daily cap) → treated as **theoretical**, excluded from the defensible total.
- **A note on terms.** ~19 providers have personal-use or proxy clauses worth a glance before you lean on them (see the [provider-terms table](#tos-attention-table)). Their access is real — we simply don't fold the **un-quantifiable** OAuth/keyless ones (e.g. `gemini-cli`, `agy`, `amazon-q` — they share quota already counted under the base provider) into the headline. None of this is legal advice; you decide.
---
## Quick reference (API key providers with `hasFree: true`)
## ToS attention table
| Provider | ID | Free tier note |
| ------------------ | --------------- | ---------------------------------------------------------------------------------------------------- |
| AgentRouter | `agentrouter` | $200 free credits on signup — multi-model routing gateway |
| AI21 Labs | `ai21` | $10 trial credits on signup (valid 3 months), no credit card required |
| AI/ML API | `aimlapi` | $0.025/day free credits — 200+ models via single endpoint |
| BazaarLink | `bazaarlink` | Free tier with `auto:free` routing — zero-cost inference, no credit card required |
| Baseten | `baseten` | $30 free trial credits for GPU inference |
| Blackbox AI | `blackbox` | Free tier: unlimited basic chat plus Minimax-M2.5, no credit card required |
| Bytez | `bytez` | $1 free credits, refreshes every 4 weeks |
| Cerebras | `cerebras` | Free: 1M tokens/day, 60K TPM — world's fastest inference |
| Cloudflare AI | `cloudflare-ai` | Free 10K Neurons/day: ~150 LLM responses or 500s Whisper audio |
| Cohere | `cohere` | Free Trial: 1,000 API calls/month for testing, no credit card required |
| Completions.me | `completions` | Free unlimited access to Claude, GPT, Gemini — no credit card, no rate limits |
| DeepInfra | `deepinfra` | Free signup credits for API testing and model exploration |
| DeepSeek | `deepseek` | 5M free tokens on signup — no credit card required |
| Enally AI | `enally` | Free for students and developers — no credit card, OTP verification |
| Fireworks AI | `fireworks` | $1 free starter credits on signup for API testing |
| FreeTheAi | `freetheai` | Community-run — free forever, no paid tiers, no credit card |
| Gemini (AI Studio) | `gemini` | Free forever: 1,500 req/day for Gemini 2.5 Flash — no credit card |
| GLHF Chat | `glhf` | Free tier for open-source model inference |
| Groq | `groq` | Free tier: 30 RPM / 14.4K RPD — no credit card |
| HuggingFace | `huggingface` | Free Inference API for thousands of models (Whisper, VITS, SDXL…) |
| Hyperbolic | `hyperbolic` | $1–5 trial credits on signup for serverless inference |
| Inference.net | `inference-net` | $25 free credits on signup plus research grants available |
| Jina AI | `jina-ai` | 10M free tokens on signup (non-commercial), no credit card required |
| Kluster AI | `kluster` | $5 free credits on signup — DeepSeek R1, Llama 4 Maverick/Scout, Qwen3 235B |
| Lepton AI | `lepton` | Free tier available — fast inference on custom hardware |
| LLM7.io | `llm7` | No signup required — 2 req/s, 20 RPM, 100 req/hr free tier |
| LongCat AI | `longcat` | 50M tokens/day (Flash-Lite) + 500K/day (Chat/Thinking) — 100% free while public beta |
| Mistral | `mistral` | Free Experiment tier: rate-limited access to all models, no credit card required |
| Modal | `modal` | $30/month free credits for new accounts |
| Morph | `morph` | Free tier: 250K credits/month, $0 |
| Nebius | `nebius` | ~$1 trial credits on signup for API testing |
| NLP Cloud | `nlpcloud` | Trial credits for new accounts |
| Nous Research | `nous-research` | Free tier: 50 RPM, 500,000 TPM — no credit card |
| Novita AI | `novita` | $0.50 trial credits on signup (valid about 1 year) |
| nScale | `nscale` | $5 free credits on signup for inference testing |
| NVIDIA NIM | `nvidia` | Free dev access: ~40 RPM, 70+ models (Kimi K2.5, GLM 4.7, DeepSeek V3.2…) |
| OpenRouter | `openrouter` | Free models at $0/token with `:free` suffix — 20 RPM / 200 RPD |
| Pollinations AI | `pollinations` | No API key required for free public endpoint. Optional Spore tier: ~0.01 pollen/hour |
| Predibase | `predibase` | $25 free trial credits (30-day validity) |
| PublicAI | `publicai` | Free community inference tier for testing |
| Puter AI | `puter` | 500+ models (GPT-5, Claude Opus 4, Gemini 3 Pro, Grok 4, DeepSeek V3…) — users pay via Puter account |
| Reka | `reka` | $10/month recurring free API credits |
| SambaNova | `sambanova` | $5 free credits on signup (30-day validity), no credit card required |
| Scaleway AI | `scaleway` | 1M free tokens for new accounts — EU/GDPR compliant (Paris), Qwen3 235B & Llama 70B |
| SiliconFlow | `siliconflow` | $1 free credits plus permanently free models after identity verification |
| Together AI | `together` | $25 signup credits + 3 permanently free models: Llama 3.3 70B, Vision, DeepSeek-R1 distill |
| UncloseAI | `uncloseai` | Free forever — no signup, no credit card. OpenAI-compatible endpoints |
| Voyage AI | `voyage-ai` | 200M free tokens for embeddings and reranking |
> A quick read on each provider's terms for a self-hosted, single-user personal proxy. `caution` = a personal-use or proxy clause worth checking; `ambiguous` = unclear; `ok` = explicitly permitted. Informational, not legal advice — you decide.
**Total: 48 API-key providers with `hasFree: true`.**
### ⚠️ Caution — personal-use / proxy clauses worth checking (19)
> All entries above are copied verbatim from the `freeNote` field in the provider catalog so they stay in sync with the code.
> Their free access is real and OmniRoute can route to them; the clauses below are just worth knowing. The OAuth/keyless ones aren't token-quantifiable, so they're not in the headline number (not because they're unusable).
| Provider | Note |
|---|---|
| `agy` | Google Antigravity ToS explicitly prohibits using third-party software, tools, or services (including proxies) to access the service via OAuth; doing… |
| `ai21` | ToS §4.2/§8.2 prohibits sublicensing or distributing API access to third parties; §3.3 restricts trial/evaluation products to "internal evaluation on… |
| `amazon-q` | Product is discontinued for new signups; existing users are subject to AWS Customer Agreement which governs use of managed services — self-hosted pro… |
| `blackbox` | ToS explicitly prohibits sublicensing, reselling, making the service available to third parties, and building derivative services — a self-hosted per… |
| `coze` | Coze ToS explicitly restricts use to "personal and non-commercial use" and prohibits renting, distributing, sublicensing, or reselling the service; a… |
| `duckduckgo-web` | Duck.ai ToS (duckduckgo.com/duckai/privacy-terms) explicitly prohibits "automated querying and developing or offering AI services" and circumventing … |
| `featherless-ai` | Individual plans explicitly restricted to "interactive use or proto-typing and experimentation by the purchaser" — inference resale and proxy use req… |
| `fireworks` | ToS explicitly prohibits proxy/intermediary use, API key transfers, and sublicensing (Sections 2.1 and 2.2(i)(j)); self-hosted personal proxies are n… |
| `friendliai` | ToS Section 8(e) and 8(f) explicitly prohibit using FriendliAI as a proxy or allowing third-party access on a standalone basis, and forbid reselling/… |
| `gemini-cli` | Google explicitly prohibits using Gemini CLI's OAuth authentication with third-party software/proxies; violations result in account bans (mass bans w… |
| `iflytek` | Section 2.4(3) of the iFlytek Spark LLM Service Agreement explicitly prohibits "using any automated or programmatic methods to extract data or output… |
| `kiro` | Kiro FAQ explicitly prohibits use with "OpenClaw and similar tools that leverage third-party harnesses" — a self-hosted AI proxy (like OmniRoute) rou… |
| `modal` | ToS Section 1.3 explicitly prohibits "rent, resell or otherwise allow any third party direct access to or use of the Service" — building a self-hoste… |
| `muse-spark-web` | Meta ToS explicitly prohibits automated access without prior permission, reverse engineering without written permission, and circumventing technologi… |
| `nlpcloud` | ToS explicitly prohibits "setting up a proxy or other device that allows others to access the Service through it" and grants only a non-transferable,… |
| `opencode` | ToS (Anomaly Innovations, Inc.) explicitly restricts use to "your own internal use, and not on behalf of or for the benefit of any third party" — ope… |
| `qwen-web` | The free OAuth tier is discontinued; no ToS permits a self-hosted proxy using session tokens against chat.qwen.ai. Even before shutdown, automated/pr… |
| `t3-web` | ToS explicitly restricts accounts to personal use only, prohibits credential sharing with third parties, and bans automated/bot/scraping access — a s… |
### ✅ Generally permissive — caution / ambiguous / ok (the rest)
| Provider | ToS | Note |
|---|---|---|
| `aimlapi` | ambiguous | ToS grants a non-exclusive use license but does not explicitly permit or prohibit self-hosted proxy or resale; no "pers… |
| `baichuan` | ambiguous | No explicit prohibition on self-hosted personal proxies found in publicly accessible docs; however, the M3 Plus free pl… |
| `bluesminds` | ambiguous | No explicit ToS clauses found regarding self-hosted proxying or resale; the pricing page focuses on feature/rate limits… |
| `bytez` | ambiguous | No explicit ToS page was accessible (404); no public evaluation-only or no-proxy clauses found in docs, but the platfor… |
| `doubao` | ambiguous | No explicit proxy/resale prohibition found in publicly indexed documentation; Volcengine is a developer-oriented cloud … |
| `gitlawb-gmi` | ambiguous | No explicit ToS clause found prohibiting self-hosted personal proxy use; the free Nemotron model carries an NVIDIA disc… |
| `inclusionai` | ambiguous | No explicit ToS found prohibiting proxy/self-hosted use, but the platform is operated by Ant Group (Chinese company) an… |
| `kluster` | ambiguous | ToS primarily covers website content rights and does not specifically address API proxy use, resale, or self-hosted pro… |
| `monsterapi` | ambiguous | MonsterAPI's ToS page (monsterapi.ai/terms-of-service) was unreachable during research; no specific proxy/resale/person… |
| `nous-research` | ambiguous | Nous Portal itself is an aggregator/proxy service; using it as a backend for another self-hosted proxy creates a proxy-… |
| `ollama-cloud` | ambiguous | ToS prohibits using the service "to develop competing products" but has no explicit ban on self-hosted personal proxies… |
| `stepfun` | ambiguous | No explicit prohibition on self-hosted personal proxy found, but the Step Plan ToS targets developers using specific co… |
| `agentrouter` | caution | No published ToS found; platform restricts accepted clients to specific AI coding tools (Claude Code, Codex, Gemini CLI… |
| `api-airforce` | caution | ToS explicitly prohibits "building competing services without permission" and "credential sharing" — a self-hosted pers… |
| `arcee-ai` | caution | Free access is via OpenRouter's :free routing layer (not Arcee's direct API terms); OpenRouter ToS permits personal dev… |
| `baidu` | caution | ToS not explicitly reviewed for proxy/resale clauses, but platform requires real-name authentication (Chinese ID typica… |
| `baseten` | caution | ToS restricts use to "Customer's internal business purposes" and explicitly prohibits sublicensing, reselling, or allow… |
| `bazaarlink` | caution | ToS explicitly prohibits reselling or sublicensing API keys to third parties; a self-hosted personal proxy for personal… |
| `brave-search` | caution | ToS prohibits redistribution, resale, and sublicensing of search results; using the API to "replicate or attempt to rep… |
| `byteplus` | caution | Tokens are non-transferable and single-account only; no explicit proxy prohibition, but BytePlus reserves the right to … |
| `cerebras` | caution | ToS grants a non-exclusive, non-transferable, non-sublicensable right for personal or business use; prohibits resale, s… |
| `cloudflare-ai` | caution | Cloudflare Self-Serve ToS §2.2.1(j) prohibits using Services to "provide a virtual private network or other similar pro… |
| `cohere` | caution | Cohere explicitly prohibits trial keys for "production or commercial purposes"; a self-hosted personal proxy routing re… |
| `deepinfra` | caution | ToS allows legal commercial use broadly, but prohibits use "directly or indirectly competitive with any business of the… |
| `deepseek` | caution | Open Platform ToS (effective 2026-04-29) permits broad use including "derivative product development" and personal/comm… |
| `dify` | caution | Self-hosted single-user personal proxy is permitted under the modified Apache 2.0 license; however, multi-tenant deploy… |
| `exa-search` | caution | No explicit "no proxy" or "evaluation only" clauses found; Exa actively offers a reseller partner program allowing API … |
| `firecrawl` | caution | Cloud API ToS has no explicit personal-proxy prohibition found, but the open-source self-hosted version is AGPL-3.0 (re… |
| `gemini` | caution | ToS explicitly states the free tier is for "developers building with Google AI models for professional or business purp… |
| `github-models` | caution | GitHub's Acceptable Use Policy prohibits reselling/proxying the service; GitHub Models ToS delegates to each model's ho… |
| `glhf` | caution | ToS explicitly prohibits sharing account credentials or making the account available to any third party, which makes a … |
| `groq` | caution | Services Agreement §6.3 prohibits reselling, sublicensing, or distributing API access; §3.2 bars reselling/leasing acco… |
| `hackclub` | caution | Service is explicitly scoped to Hack Club teen members building projects/learning; no public ToS found explicitly permi… |
| `huggingchat` | caution | Hugging Face ToS does not explicitly ban personal self-hosted proxies, but supplemental terms (referenced but not fully… |
| `huggingface` | caution | ToS grants a limited license to access/use the service; the document does not explicitly permit or forbid a single-user… |
| `hyperbolic` | caution | ToS grants API access "solely for your own personal or internal business purposes" and explicitly prohibits licensing, … |
| `inference-net` | caution | ToS explicitly prohibits "sublicense, resell, distribute" and transferring API keys without written consent; a single-u… |
| `jina-ai` | caution | Free 10M tokens are explicitly non-commercial (CC-BY-NC 4.0 model license); a single-user personal proxy for personal L… |
| `jina-reader` | caution | ToS prohibits using outputs to build competing services and bans "automated methods to extract information via scraping… |
| `llm7` | caution | ToS positions the service as for "experimentation, development, and research"; no explicit ban on self-hosted personal … |
| `longcat` | caution | The API Platform Service Agreement (longcat.chat/platform/private/) permits commercial integration and self-hosted apps… |
| `mistral` | caution | Consumer ToS explicitly states APIs may only be used for "personal needs" and prohibits making API keys available to th… |
| `morph` | caution | ToS allows commercial use generally; self-hosted proxy deployments require explicit arrangement with sales. Section 18.… |
| `nebius` | caution | ToS (Section 5f) explicitly prohibits resale, redistribution, or offering the service "on a standalone basis" — a self-… |
| `nomic` | caution | ToS grants a non-exclusive, non-transferable API license; Section 6.b prohibits building a competitive service. Using t… |
| `novita` | caution | ToS prohibits resale and competing services but does not explicitly address personal self-hosted proxies; personal use … |
| `nscale` | caution | AUP prohibits "copy, modify, duplicate... frame, mirror, republish... distribute all or any part of the Nscale Platform… |
| `nvidia` | caution | Free tier is explicitly for prototyping/dev/research/evaluation only — production use (serving real end-users) requires… |
| `openrouter` | caution | ToS explicitly prohibits reselling API access or developing a competing service; single-user self-hosted personal proxy… |
| `pollinations` | caution | MIT License cited in API docs suggests liberal reuse; no explicit prohibition on self-hosted proxying found. However, u… |
| `predibase` | caution | Predibase is positioned as an enterprise fine-tuning/serving platform; the free trial is explicitly for exploration and… |
| `publicai` | caution | ToS (publicai.co/tc) designates services as "primarily for research and educational use"; no explicit proxy or resale p… |
| `puter` | caution | Puter ToS forbids using services for "commercial purpose" without written consent; a self-hosted personal proxy consumi… |
| `qoder` | caution | ToS page returned no readable content; Qoder is a coding IDE client (not a public API), and third-party proxy wrappers … |
| `reka` | caution | Business Terms prohibit sublicensing or distributing access to third parties; a personal single-user proxy is likely fi… |
| `sambanova` | caution | ToS Section 1.5(c) explicitly prohibits reselling, sublicensing, or making the service available to third parties; a se… |
| `sensenova` | caution | No explicit proxy or resale prohibition found in reviewed ToS, but the free tier is a promotional beta with no SLA, Sen… |
| `serper-search` | caution | ToS explicitly prohibits "mirroring materials on any other server as-is with no-value-added" — a simple pass-through pr… |
| `siliconflow` | caution | ToS (Clause 3.4(e)(f)(p)) explicitly prohibits making the service available to any third party, reselling/sublicensing,… |
| `sparkdesk` | caution | SparkDesk User Agreement grants only personal, non-commercial use rights; API Interface Policy prohibits automated data… |
| `tavily-search` | caution | ToS explicitly states the API "may not be transferred, assigned, shared, or otherwise made available to any third party… |
| `tencent` | caution | Tencent Cloud ToS explicitly prohibits sublicensing or reselling API access; a self-hosted personal proxy for personal … |
| `together` | caution | ToS Section 4.3(d) explicitly prohibits transferring, distributing, reselling, leasing, or offering the Services on a s… |
| `uncloseai` | caution | Personal proxy use is plausible but not explicitly permitted; ToS bans building "competing machine learning services wi… |
| `veoaifree-web` | caution | ToS explicitly bans automated bots or scripts running at "inhuman speeds" and prohibits copying the platform to create … |
| `vertex` | caution | Google Cloud Service Terms restrict resale to authorized resellers only (Section 14 requires a Reseller Agreement); a s… |
| `voyage-ai` | caution | ToS grants "personal, non-commercial use" for site content and prohibits credential/account sharing with third parties;… |
| `360ai` | unknown | ToS for developer API not publicly accessible without registration; access requires application approval which implies … |
| `chutes` | unknown | ToS page exists at chutes.ai/terms but content was not accessible via fetch; no explicit proxy/resale clauses found in … |
| `freemodel-dev` | unknown | The Terms of Service page (freemodel.dev/terms) returned only a header with no readable content via WebFetch; no clause… |
| `gitlawb` | unknown | No ToS or acceptable-use policy found; proxy/resale restrictions unknown — assume caution for self-hosted proxy use. |
| `liquid` | unknown | No hosted API exists to proxy; open-source model commercial use is free for orgs under $10M annual revenue. No self-hos… |
| `phind` | unknown | Service is discontinued; ToS is no longer relevant. No proxy use is possible. |
| `theoldllm` | unknown | No terms of service document was found on the site; proxying, resale, or self-hosted use policy is entirely undocumente… |
| `yi` | unknown | ToS not publicly accessible without login; no proxy/resale clauses could be reviewed. Self-hosted personal proxy use st… |
| `comfyui` | ok | GPL-3.0 open-source license explicitly permits self-hosted personal proxy use; Comfy Org ToS confirms commercial use of… |
| `scaleway` | ok | Scaleway's General Terms of Services are a standard commercial cloud agreement with no explicit prohibition on self-hos… |
| `sdwebui` | ok | AGPL-3.0 license: free to self-host for personal use with no restrictions on usage volume; a personal proxy using this … |
| `searxng-search` | ok | AGPL-3.0 open-source license explicitly permits self-hosted personal proxy use with no restriction on usage type, resal… |
---
## OAuth-based free tiers
## Per-provider free-tier (current, researched 2026-06-05)
### Always-free OAuth providers (in `FREE_PROVIDERS`)
> Sorted by estimated recurring monthly free tokens. `—` = not token-quantifiable (credits / one-time / search / image / discontinued). `conf` = research confidence.
These providers are designed around a vendor OAuth flow and ship a free tier by default:
| Provider | ID | Notes |
| ---------- | ------------ | ----------------------------------------------------------------------------------------------------------- |
| Qoder AI | `qoder` | OAuth or Personal Access Token. Free tier on signup. |
| Gemini CLI | `gemini-cli` | Uses Gemini CLI OAuth / Cloud Code credentials. Pro models require an eligible Google account or paid plan. |
| Kiro AI | `kiro` | AWS Builder ID (Kiro Free tier). |
| Amazon Q | `amazon-q` | Same AWS Builder ID / refresh-token flow as Kiro, but kept as separate connections. |
### OAuth providers with vendor-controlled free tiers (in `OAUTH_PROVIDERS`)
The free-tier surface here depends entirely on each vendor's account plan, not on OmniRoute:
| Provider | ID | Auth hint |
| -------------------- | ------------- | --------------------------------------------------------------------------------------- |
| Claude Code | `claude` | OAuth via `platform.claude.com`. Free quota depends on your Anthropic account. |
| Antigravity | `antigravity` | Google OAuth (Antigravity). |
| OpenAI Codex | `codex` | OAuth via OpenAI (Codex CLI). Subject to ChatGPT plan free credits. |
| GitHub Copilot | `github` | OAuth via GitHub. Free for verified students; free trial otherwise. |
| GitLab Duo | `gitlab-duo` | OAuth (`ai_features + read_user`). Requires GitLab Duo entitlement. |
| Cursor IDE | `cursor` | Cursor OAuth. Free tier limits depend on Cursor plan. |
| Kimi Coding | `kimi-coding` | Moonshot OAuth. Free quota on Kimi Coding accounts. |
| Kilo Code | `kilocode` | Kilo OAuth — free auto-router available. |
| Cline | `cline` | Cline OAuth. |
| Windsurf (Devin CLI) | `windsurf` | Sign in at `windsurf.com`, paste your token. Free tier limits set by Windsurf. |
| Devin CLI (Official) | `devin-cli` | Uses the official Devin CLI binary or `WINDSURF_API_KEY`. Subject to Devin's free tier. |
| Provider | Category | Free type | Est. tokens/mo | Conf | ToS | Current status |
|---|---|---|---|---|---|---|
| `sparkdesk` | llm-chat | keyless-limited | 2.59B | med | caution | Spark Lite model is permanently free with a rate limit of 2 QPS (approximately 120 RPM) per App ID and no documented to… |
| `tencent` | llm-chat | keyless-limited | 2.07B | med | caution | Hunyuan-lite is permanently free (no token quota, no expiry) as of May 2026, subject only to a default 5-concurrent-ses… |
| `siliconflow` | aggregator | recurring-monthly | 1.73B | med | caution | SiliconFlow provides $1 one-time trial credits for new account signups plus a set of permanently free (priced at $0/tok… |
| `nvidia` | llm-chat | keyless-limited | 1.38B | med | caution | NVIDIA NIM offers a permanent free API key (no credit card required) with access to 70–100+ hosted models on build.nvid… |
| `mistral` | llm-chat | recurring-monthly | 1.00B | high | caution | Mistral offers a free "Experiment" tier with no credit card required (phone verification only), granting access to all … |
| `baidu` | llm-chat | keyless-limited | 864M | med | caution | ERNIE-Speed-8K/128K, ERNIE-Lite-8K/128K, and ERNIE-Tiny are permanently free (since May 21, 2024) for all users who com… |
| `publicai` | llm-chat | keyless-limited | 691M | med | caution | PublicAI offers a free API (OpenAI-compatible) via platform.publicai.co with no published token cap, rate-limited to 20… |
| `longcat` | llm-chat | recurring-daily | 150M | med | caution | LongCat API Platform (public beta by Meituan) provides 5,000,000 free tokens per day for the LongCat-2.0-Preview model;… |
| `cloudflare-ai` | llm-chat | recurring-daily | 122M | high | caution | Cloudflare Workers AI provides 10,000 Neurons per day free on both Free and Paid Workers plans, resetting daily at 00:0… |
| `doubao` | llm-chat | recurring-daily | 60M | med | ambiguous | Volcengine Ark offers two free tiers for Doubao API: a one-time 500K tokens/model welcome quota (30-day validity, real-… |
| `gemini` | llm-chat | recurring-daily | 60M | med | caution | Google AI Studio free tier (as of mid-2026) provides recurring free access with no credit card required, but limits wer… |
| `cerebras` | llm-chat | recurring-daily | 30M | med | caution | Cerebras offers a recurring free "Free Trial" tier with 1M tokens/day, 5 RPM, and 30K TPM (per-model basis on current d… |
| `api-airforce` | aggregator | recurring-daily | 24M | med | caution | Api.airforce offers a free registered-account tier limited to 1 request/minute and 1,000 requests/day with access to ba… |
| `ollama-cloud` | llm-chat | recurring-monthly | 20M | med | ambiguous | Ollama Cloud offers a free tier ($0/month) with "light usage" access to cloud-hosted open models; usage is GPU-time-bas… |
| `github-models` | aggregator | recurring-daily | 18M | high | caution | All GitHub accounts get free daily rate-limited access to 160+ models including GPT-5, o-series, DeepSeek-R1, Grok-3, a… |
| `groq` | llm-chat | recurring-daily | 15M | high | caution | Groq offers a free tier with no credit card required, providing 30 RPM and per-model daily caps (up to 14.4K RPD / 500K… |
| `inclusionai` | llm-chat | recurring-daily | 15M | med | ambiguous | InclusionAI (via developer.ant-ling.com, the official Ant Group API portal) provides a free tier of 500,000 tokens/day … |
| `bluesminds` | aggregator | recurring-daily | 7M | med | ambiguous | BluesMinds offers a permanent free plan with 500 pi credits on signup, 20 RPM, and 300 requests/day, with access limite… |
| `sambanova` | llm-chat | recurring-daily | 6M | med | caution | SambaNova offers a permanent recurring free tier (no credit card required) with 20 RPM, 20 RPD, and 200,000 TPD. New si… |
| `arcee-ai` | llm-chat | keyless-limited | 5M | med | caution | Arcee AI offers free access to Trinity Large Preview and Trinity Large Thinking via OpenRouter's :free tier (20 RPM, ~2… |
| `llm7` | llm-chat | keyless-limited | 4M | med | caution | LLM7.io offers a free tier that requires obtaining a free token from token.llm7.io (light signup). The authenticated fr… |
| `bazaarlink` | aggregator | recurring-daily | 4M | med | caution | BazaarLink offers a permanent free tier with no credit card required, featuring the auto:free model that routes to zero… |
| `openrouter` | aggregator | recurring-daily | 1M | high | caution | Free models (`:free` suffix) are available at $0/token with 20 RPM and 50 RPD for accounts without purchased credits; t… |
| `cohere` | llm-chat | recurring-monthly | 800K | high | caution | Cohere offers a free Trial API key (auto-issued on signup) limited to 1,000 API calls/month across all endpoints, with … |
| `huggingchat` | llm-chat | recurring-monthly | 500K | med | caution | HuggingChat free tier provides $0.10/month in Hugging Face Inference Provider credits (recurring monthly). The previous… |
| `morph` | llm-code | recurring-monthly | 400K | med | caution | Morph offers a free tier with 200 requests/month and 250K credits, described as intended for testing and personal proje… |
| `huggingface` | aggregator | recurring-monthly | 200K | high | caution | Free HuggingFace accounts receive $0.10/month in recurring Inference Provider credits (explicitly "subject to change") … |
| `kiro` | llm-code | recurring-monthly | 25K | high | caution | Kiro AI offers a perpetual free tier of 50 credits/month (recurring, resets each billing cycle, unused credits do not c… |
| `360ai` | llm-chat | one-time-trial-credit | — | low | unknown | 360 AI (360智脑) API requires formal application and approval to access. New users historically received a one-time promo… |
| `agentrouter` | aggregator | one-time-trial-credit | — | low | caution | AgentRouter offers a one-time credit on signup ($100 for standard, $200 via referral) to use across 30+ LLM providers v… |
| `agy` | llm-code | account-oauth | — | med | caution | Antigravity CLI (agy) offers a free tier requiring OAuth login with a personal Google account, with a weekly-based rate… |
| `ai21` | llm-chat | one-time-trial-credit | — | med | caution | AI21 Labs offers new accounts $10 in trial credits valid for 7 days (no credit card required); after expiry, pay-as-you… |
| `aimlapi` | aggregator | discontinued | — | high | ambiguous | The free tier is officially paused as of mid-2025 (docs last updated ~May 2026). AI/ML API now operates exclusively on … |
| `amazon-q` | llm-code | discontinued | — | high | caution | Amazon Q Developer is discontinued for new signups as of May 15, 2026 — both free tier and paid subscriptions can no lo… |
| `baichuan` | llm-chat | one-time-trial-credit | — | med | ambiguous | Baichuan operates on a pay-as-you-go model with a one-time 80 CNY (~$11 USD) trial credit for new accounts (valid 3 mon… |
| `baseten` | other | one-time-trial-credit | — | med | caution | Baseten offers $30 one-time trial credits for new workspaces on the Startup (Basic) plan. After those credits are used,… |
| `blackbox` | llm-code | keyless-limited | — | med | caution | Blackbox AI offers a free web/IDE tier with unlimited basic chat but restricts advanced models (GPT-4o, Claude, etc.) t… |
| `brave-search` | search | recurring-credit | — | high | caution | As of February 12, 2026, Brave removed its previously free (5,000 queries/month, no card required) plan and replaced it… |
| `byteplus` | llm-chat | one-time-trial-credit | — | high | caution | BytePlus ModelArk provides new users a one-time free trial of 500,000 tokens per LLM model (2,000,000 for vision models… |
| `bytez` | aggregator | recurring-credit | — | med | ambiguous | Bytez offers $1 in free credits that refresh every 4 weeks (credits expire if unused within the cycle). Free tier is li… |
| `chutes` | aggregator | discontinued | — | high | unknown | The free Early Access program (200 requests/day) was officially discontinued on March 15, 2026. Chutes.ai now operates … |
| `comfyui` | image | keyless-unlimited | — | high | ok | ComfyUI is a fully open-source (GPL-3.0), self-hosted diffusion model interface that runs entirely on local hardware wi… |
| `coze` | aggregator | recurring-daily | — | med | caution | Coze's free plan provides 10 message credits per day — a platform-level unit (not raw LLM tokens) where each model call… |
| `deepinfra` | aggregator | one-time-trial-credit | — | med | caution | DeepInfra is a pay-as-you-go inference provider that explicitly requires a credit card or prepayment to use services; a… |
| `deepseek` | llm-chat | one-time-trial-credit | — | high | caution | DeepSeek offers a one-time signup credit of 5 million tokens (no credit card required) valid for 30 days from account c… |
| `dify` | other | one-time-trial-credit | — | med | caution | Dify Cloud offers a Sandbox (free) plan with 200 message credits (one-time trial for testing with bundled LLM keys), 5 … |
| `duckduckgo-web` | llm-chat | keyless-limited | — | high | caution | Duck.ai is free and keyless (no account or API key required), with anonymous daily usage limits that DuckDuckGo deliber… |
| `exa-search` | search | recurring-monthly | — | high | caution | Exa offers a permanently free plan with 1,000 search requests per month (no expiration), with contents (text and highli… |
| `featherless-ai` | llm-chat | discontinued | — | high | caution | Featherless AI has no free tier or free trial for general users as of June 2026. Paid subscriptions start at $10/month … |
| `firecrawl` | tool | recurring-monthly | — | high | caution | Firecrawl offers a free plan with 1,000 credits per month (1 credit = 1 page scraped), 2 concurrent requests, and low r… |
| `fireworks` | llm-chat | one-time-trial-credit | — | high | caution | Fireworks AI offers $1 in one-time starter credits on signup; no recurring free tier exists. Without a payment method o… |
| `freemodel-dev` | llm-chat | one-time-trial-credit | — | low | unknown | FreeModel.dev (domain registered April 30, 2026) offers $300 in one-time free credits on signup with no payment info re… |
| `friendliai` | llm-chat | keyless-limited | — | med | caution | FriendliAI offers a Tier 0 account for new signups with adaptive (dynamically throttled) rate limits and 8K max output … |
| `gemini-cli` | llm-chat | account-oauth | — | high | caution | Gemini CLI's free tier (Google Account OAuth, 1,000 req/day, 60 RPM) is being deprecated on June 18, 2026 for all non-e… |
| `gitlawb` | aggregator | unknown | — | low | unknown | The original free MiMo (xiaomi/mimo-v2.5) model was revoked server-side around May 24, 2026. As of June 2026 the platfo… |
| `gitlawb-gmi` | llm-chat | keyless-limited | — | med | ambiguous | As of June 2026, Gitlawb Opengateway is primarily a pay-as-you-go credit-balance gateway; the only recurring-free model… |
| `glhf` | llm-chat | one-time-trial-credit | — | med | caution | GLHF Chat ended its free beta in January 2025 and moved to pay-as-you-go pricing (per-token, no subscription required).… |
| `hackclub` | aggregator | account-oauth | — | med | caution | Free AI API access for Hack Club members via OAuth sign-in; no public rate limit numbers documented. Provides 30+ model… |
| `hyperbolic` | llm-chat | one-time-trial-credit | — | med | caution | Hyperbolic gives new users $1 in one-time trial credits on signup, with a Basic plan rate limit of 60 RPM; upgrading to… |
| `iflytek` | llm-chat | keyless-limited | — | med | caution | Spark Lite remains permanently free as of June 2026, with unlimited tokens but a 2 QPS (≈120 RPM) rate limit per App ID… |
| `inference-net` | llm-chat | recurring-monthly | — | med | caution | Inference.net currently offers a free plan ($0 forever) with $1 in recurring monthly credits that can be spent on pay-a… |
| `jina-ai` | search | one-time-trial-credit | — | med | caution | New API keys receive 10 million free tokens (one-time, non-commercial) usable across all Jina AI endpoints (embeddings,… |
| `jina-reader` | web-reverse | keyless-limited | — | med | caution | Jina Reader is freely accessible without any API key at 20 RPM (keyless, rate-limited by IP). New accounts registering … |
| `kluster` | llm-chat | one-time-trial-credit | — | med | ambiguous | New users receive $5 in free credits on signup and email verification; multiple sources also indicate a permanent free … |
| `liquid` | llm-chat | unknown | — | high | unknown | Liquid AI does not currently offer a hosted API of their own; models are open-source and available via Hugging Face, LE… |
| `modal` | other | recurring-monthly | — | high | caution | Modal's Starter plan gives every account $30/month in recurring free compute credits (GPU + CPU per-second billing), wi… |
| `monsterapi` | llm-chat | one-time-trial-credit | — | low | ambiguous | MonsterAPI offers a free tier that gives new users one-time trial credits upon signup (no credit card required), but th… |
| `muse-spark-web` | llm-chat | account-oauth | — | high | caution | Meta AI at meta.ai is free for any user with a Meta/Facebook account, with no published hard rate or token limits for c… |
| `nebius` | llm-chat | one-time-trial-credit | — | med | caution | Nebius AI Studio (Token Factory) offers ~$1 in one-time trial credits to new signups with no credit card required, givi… |
| `nlpcloud` | llm-chat | recurring-monthly | — | med | caution | NLP Cloud offers a permanent recurring free plan with 10,000 API requests per month at up to 3 requests per minute, wit… |
| `nomic` | embeddings | one-time-trial-credit | — | med | caution | Nomic offers a one-time free allowance of 1 million tokens for the Embed API; after that, a paid subscription is requir… |
| `nous-research` | aggregator | recurring-credit | — | med | ambiguous | Nous Portal (launched April 27, 2026) offers a free tier at $0/month with $0.10 in monthly recurring credits, plus perm… |
| `novita` | aggregator | one-time-trial-credit | — | med | caution | Novita AI provides $0.50 one-time trial credits upon signup with a 60 RPM rate limit; no recurring free tier exists, th… |
| `nscale` | llm-chat | one-time-trial-credit | — | med | caution | nScale offers $5 in free credits to every new user signing up for their serverless inference API; this is a one-time tr… |
| `opencode` | llm-code | keyless-limited | — | med | caution | OpenCode is an open-source AI coding agent (client tool); its companion hosted service "OpenCode Go" offers a free tier… |
| `phind` | llm-code | discontinued | — | high | unknown | Phind permanently shut down on January 16, 2026, without advance notice, just over a month after raising $10M in fundin… |
| `pollinations` | aggregator | keyless-limited | — | med | caution | Pollinations AI provides a keyless public API (no signup required for basic access) for image, text, audio, and video g… |
| `predibase` | llm-code | one-time-trial-credit | — | med | caution | Predibase was acquired by Rubrik in June/July 2025 and its main domain now redirects to Rubrik marketing pages. The pro… |
| `puter` | aggregator | account-oauth | — | med | caution | Puter provides 500+ AI models (GPT, Claude, Gemini, Llama, DeepSeek, Grok, etc.) via an OpenAI-compatible endpoint at a… |
| `qoder` | llm-code | one-time-trial-credit | — | med | caution | Qoder offers a free Community Edition (launched April 30, 2026) that includes unlimited code completions/next-edit sugg… |
| `qwen-web` | llm-chat | discontinued | — | high | caution | The Qwen OAuth free tier (which powered token-based API access to chat.qwen.ai) was discontinued on April 15, 2026. New… |
| `reka` | llm-chat | recurring-monthly | — | med | caution | Reka offers $10/month in recurring free API credits (refreshed at the start of each month) usable across all API featur… |
| `scaleway` | llm-chat | one-time-trial-credit | — | med | ok | New Scaleway accounts receive 1,000,000 free tokens (plus 60 minutes of audio transcription) as a one-time trial credit… |
| `sdwebui` | image | keyless-unlimited | — | high | ok | AUTOMATIC1111 Stable Diffusion WebUI is a free, open-source (AGPL-3.0) self-hosted web UI for Stable Diffusion image ge… |
| `searxng-search` | search | keyless-unlimited | — | high | ok | SearXNG is free, open-source (AGPL-3.0) self-hosted metasearch software — there is no hosted SaaS API tier or pricing m… |
| `sensenova` | llm-chat | one-time-trial-credit | — | med | caution | As of June 2026, SenseNova offers a limited-time free public beta ("Token Plan") giving developers 1,500 API calls per … |
| `serper-search` | search | one-time-trial-credit | — | high | caution | Serper offers 2,500 free queries as a one-time trial with no credit card required. These credits do not renew — after e… |
| `stepfun` | llm-chat | one-time-trial-credit | — | med | ambiguous | StepFun's platform (platform.stepfun.ai / platform.stepfun.com) no longer offers free LLM model access — all LLM API ca… |
| `t3-web` | aggregator | recurring-daily | — | med | caution | t3.chat offers a free tier with limited daily messages (exact count undisclosed) across a restricted set of models, res… |
| `tavily-search` | search | recurring-monthly | — | high | caution | Tavily offers 1,000 free API credits per month with no credit card required. Credits cover basic search (1 credit each)… |
| `theoldllm` | llm-chat | keyless-unlimited | — | low | unknown | The Old LLM is a keyless, no-signup web chat UI hosted on Vercel that claims unlimited free access to 60+ AI models, bu… |
| `together` | llm-chat | one-time-trial-credit | — | med | caution | Together AI offers a one-time trial credit (reported as $25 by third-party aggregators, though official billing docs sa… |
| `uncloseai` | llm-chat | keyless-unlimited | — | med | caution | UncloseAI remains a completely free, no-signup, keyless LLM service serving Hermes-3-Llama-3.1-8B and Qwen 3 Coder via … |
| `veoaifree-web` | video | keyless-unlimited | — | med | caution | Veoaifree.com offers completely keyless, no-login video generation (VEO 3.1, VEO 2.0, Seedance 2.0) with no documented … |
| `vertex` | llm-chat | one-time-trial-credit | — | high | caution | Vertex AI is a pay-as-you-go enterprise Google Cloud service with no recurring free inference tier. New GCP accounts re… |
| `voyage-ai` | embeddings | one-time-trial-credit | — | high | caution | Voyage AI provides a one-time free allocation of 200M tokens per account for most current embedding and reranking model… |
| `yi` | llm-chat | unknown | — | low | unknown | The Yi API platform (platform.01.ai) appears to be pay-as-you-go only with no publicly documented free tier; Yi-Lightni… |
---
## Deprecated / discontinued
## What changed since the shipped catalog (`freeNote`)
### Qwen Code (`qwen`)
> The v3.8.0-era `freeNote` strings are stale. Corrections found by this research (these drive the catalog update in `_tasks/features-v3.8.12`):
Marked `deprecated: true` in `FREE_PROVIDERS`. Discontinued **2026-04-15**.
> Qwen OAuth free tier was discontinued on 2026-04-15. Use `bailian-coding-plan`, `alibaba`, `alibaba-cn`, or `openrouter` providers with an API key instead.
Connections of type `qwen` will keep working until their tokens expire, but no new OAuth sign-ins are accepted upstream. Migrate to:
- `bailian-coding-plan` (Alibaba Coding Plan — Claude-compatible)
- `alibaba` (Alibaba — DashScope international)
- `alibaba-cn` (Alibaba (China) — DashScope China)
- `openrouter` (Qwen models exposed via OpenRouter)
---
## Command Code
`command-code` is a separate API-key provider for the Command Code agent (see `commandcode.ai`). It is not flagged with `hasFree: true` in the catalog, so it does not appear in the free table above, but it is included here because it ships in v3.8.0 alongside the free-tier providers:
- ID: `command-code`
- Endpoint: Command Code `/alpha/generate`
- Auth: Bearer API key, configured from the dashboard.
Check Command Code's website for the current free-tier policy.
---
## Environment variables
OmniRoute v3.8.2 does **not** read provider API keys from environment variables (with one exception below). Keys are stored in the encrypted SQLite database and configured from the dashboard. The env vars listed here are the only ones that affect free-tier behavior:
```bash
# Windsurf / Devin CLI — Firebase Web API key used by the Secure Token
# Service to refresh the Windsurf app. The default value ships in
# .env.example (it is a public Firebase Web API key extracted from the
# Devin CLI binary, not a real secret); override only if you mirror your
# own Windsurf token-refresh service.
WINDSURF_FIREBASE_API_KEY=<see .env.example>
# Optional fallback for the devin-cli executor when no connection key is set.
WINDSURF_API_KEY=
# Optional path to the official Devin CLI binary.
CLI_DEVIN_BIN=/usr/local/bin/devin
# OAuth client overrides (rarely needed — defaults shipped in code)
CODEX_OAUTH_CLIENT_ID=
GEMINI_OAUTH_CLIENT_ID=
GEMINI_OAUTH_CLIENT_SECRET=
GEMINI_CLI_OAUTH_CLIENT_ID=
GEMINI_CLI_OAUTH_CLIENT_SECRET=
QWEN_OAUTH_CLIENT_ID=
KIMI_CODING_OAUTH_CLIENT_ID=
GITHUB_OAUTH_CLIENT_ID=
GITLAB_DUO_OAUTH_CLIENT_ID=
GITLAB_DUO_OAUTH_CLIENT_SECRET=
QODER_OAUTH_CLIENT_SECRET=
QODER_PERSONAL_ACCESS_TOKEN=
# CLI sidecar binaries
CLI_CODEX_BIN=codex
CLI_CURSOR_BIN=agent
CLI_CLINE_BIN=cline
CLI_QODER_BIN=qoder
CLI_QWEN_BIN=qwen
```
For all other providers (Groq, Cerebras, Mistral, Gemini, Cohere, NVIDIA, OpenRouter, Together, Fireworks, Cloudflare AI, SambaNova, HuggingFace, SiliconFlow, Hyperbolic, Morph, LLM7, Lepton, Kluster, UncloseAI, BazaarLink, Completions, Enally, FreeTheAi, AgentRouter, Command Code, etc.), add the key from `/dashboard/providers/new`.
---
## How to use
1. Open `/dashboard/providers/new` and pick the provider you want.
2. Paste the API key (or complete the OAuth flow). For OAuth providers, follow the dashboard wizard.
3. The provider appears in your routing pool automatically and is eligible for combos and auto-routing.
4. Track usage at `/dashboard/usage` to see how close you are to free-tier limits.
### Suggested combos
| Goal | Strategy | Notes |
| ---------------------------- | -------------------- | --------------------------------------------------------------------- |
| Cheapest possible chat | `auto/cheap` | Prefers free / lowest-cost providers; falls back automatically. |
| Local-only routing | `auto/offline` | Routes only to local providers (Ollama, LM Studio, vLLM, …). |
| Redundancy across free tiers | combo `priority` | List Groq → Cerebras → Mistral → Gemini → NVIDIA → OpenRouter. |
| High RPM throughput | combo `round-robin` | Spreads requests across all configured free providers. |
| Best success rate | combo `lkgp` / `p2c` | Picks last-known-good provider or "power of two choices" rebalancing. |
### Tips
- Combine multiple free providers in a combo (`/dashboard/combos`) to maximize daily quota and route around outages.
- Use `omniroute doctor` to verify all configured free providers are reachable.
- Check provider health in `/dashboard/monitoring/health` — a provider with an open circuit breaker is skipped automatically.
- Free-tier limits change frequently; the `freeNote` strings reflect the limits as known at v3.8.0 ship date. Verify with each provider's official docs before relying on a specific number.
- **`360ai`** — The shipped freeNote "Free 360 AI Brain models" appears outdated. Current access is application-gated and paid. The 2023 launch-era promotional tokens (100M–250M one-time) may have been the basis for…
- **`agentrouter`** — Our shipped freeNote says "$200 free credits on signup." Current reality shows standard (non-referral) signups receive only $100; referral signups may get $200 but a community comment from April 2026…
- **`agy`** — Our shipped freeNote says "(none)" implying no free tier, but Antigravity does have a free OAuth-gated tier. However, the ToS explicitly prohibits using this free tier through a proxy like OmniRoute …
- **`ai21`** — Tightened: trial window shrunk from "3 months" to 7 days. The $10 credit amount remains the same, but validity dropped sharply from ~90 days to 7 days.
- **`aimlapi`** — Changed significantly. Shipped freeNote advertised "$0.025/day free credits — 200+ models" but the free tier is now paused/discontinued. The $0.025/day credit allocation (50,000 credits/day, 10 req/d…
- **`amazon-q`** — Our shipped freeNote says "(none)" — the reality is worse: the product is now discontinued for new signups (May 15, 2026). Previously the free tier offered 50 agentic requests/month + unlimited inlin…
- **`api-airforce`** — Catalog ships freeNote "(none)" but a documented free tier exists: 1 RPM / 1,000 RPD recurring, account signup required, limited to basic models.
- **`arcee-ai`** — The shipped freeNote ("Free Trinity Large Thinking model (262K context)") is partially accurate — Trinity Large Thinking is indeed free via OpenRouter with 262K context — but the note omits that this…
- **`baichuan`** — Our shipped freeNote says "Free Baichuan models" which implies ongoing free access, but current reality is only a one-time 80 CNY trial credit for new users (valid 3 months). There are no permanently…
- **`baidu`** — The catalog says "Free ERNIE Speed/Lite models" which is broadly accurate, but understates the scope: ERNIE-Tiny and multiple context-window variants (8K and 128K) are also free. The free tier appear…
- **`bazaarlink`** — Broadly matches — the shipped freeNote accurately describes auto:free routing for zero-cost inference. However, the current reality includes explicit rate limits (10-20 RPM, ~150 RPD) not mentioned i…
- **`blackbox`** — Our shipped freeNote claims "unlimited basic chat plus Minimax-M2.5." In reality, unlimited Minimax-M2.5 agent requests are a paid-plan feature (Pro+), not part of the free tier. The free tier has li…
- **`bluesminds`** — Our shipped freeNote was "(none)" — but BluesMinds does have a documented free tier: 500 pi credits, 20 RPM, 300 RPD, permanent free plan. The catalog significantly understates the offering.
- **`brave-search`** — The catalog notes "(none)" suggesting no free tier was tracked, but in reality there was a free 5,000 queries/month tier (no card) until February 12, 2026, which has since been replaced by a $5/month…
- **`byteplus`** — Our catalog shipped "(none)" but BytePlus ModelArk does have a free tier: a one-time trial credit of 500k tokens per LLM model for new accounts. The catalog underreports this.
- **`cerebras`** — TPM appears tightened from 60K to 30K on current documented models (gpt-oss-120b, zai-glm-4.7). RPM of 5 is now explicitly documented (was not in our shipped note). Daily token cap of 1M/day is uncha…
- **`chutes`** — The shipped freeNote says "Free tier available" but as of March 15, 2026, the free tier has been officially discontinued. The catalog note is stale and should be updated to reflect that there is no r…
- **`coze`** — The shipped note "Free ByteDance agent platform" is directionally accurate but omits that the free tier is now tightly credit-capped (10 credits/day ≈ 5–100 messages depending on model), a constraint…
- **`deepinfra`** — Our shipped freeNote says "Free signup credits for API testing" — this appears stale. The official pricing page now requires card/prepayment with no documented general free signup credit. The free ti…
- **`deepseek`** — Our shipped note says "5M free tokens on signup - no credit card required" — this is still accurate for the one-time grant, but importantly the credits expire after 30 days (not mentioned in the ship…
- **`dify`** — The shipped freeNote ("Free open-source AI app builder + RAG") is directionally accurate but incomplete. The cloud free tier is more constrained than implied: 200 message credits appear to be a one-t…
- **`doubao`** — The shipped freeNote "Free Doubao models (ByteDance)" is directionally correct but underspecified. Current reality is more structured: there is a quantified recurring daily free tier (2M tokens/day v…
- **`duckduckgo-web`** — The core "free anonymous access" description still holds, but the service has matured significantly: it now has explicit paid tiers (Plus/Pro) with higher limits implying the free tier is rate-constr…
- **`exa-search`** — Catalog ships freeNote "(none)" but Exa has a documented recurring free tier of 1,000 requests/month. This is a significant gap — the free tier exists and is permanent.
- **`featherless-ai`** — Our shipped freeNote says "Free tier available" but there is no general free tier. The only free access is through an invitation/application-based Builder Series program, which is not a standard free…
- **`firecrawl`** — Our catalog shipped freeNote "(none)", implying no free tier. In reality Firecrawl has a documented recurring free plan with 1,000 credits/month — the catalog entry is incorrect.
- **`freemodel-dev`** — Our shipped freeNote is "(none)" — this was likely a placeholder meaning the provider was not yet cataloged. In reality the provider does have a $300 one-time trial credit offer. However, this is a o…
- **`friendliai`** — The shipped freeNote ("Free tier for serverless inference") is partially accurate but misleading. There is free access via Tier 0 and free-designated models, but the rate limits are undefined and ada…
- **`gemini`** — The shipped freeNote says "1,500 req/day for Gemini 2.5 Flash" — this was accurate before December 2025. Google cut free-tier limits by 50-80% in December 2025, reducing Gemini 2.5 Flash from 1,500 R…
- **`gemini-cli`** — Catalog ships "(none)" implying no free tier was recognized. In reality, Gemini CLI did have a notable free OAuth tier (1,000 RPD via Google Account) until recently, but it is now being shut down (Ju…
- **`github-models`** — Catalog note "Free GPT-5, o-series, DeepSeek-R1, Llama 4, Grok 3" is directionally correct about model availability but omits the daily rate limits (50 RPD for high-tier models, 150 RPD for low-tier)…
- **`gitlawb`** — The shipped freeNote "Free tier available" is effectively stale. The original free MiMo access was removed in May 2026; the only remaining "free" option is a temporary promotional model (Nemotron 3 U…
- **`gitlawb-gmi`** — Partially still accurate — free tier exists but is now narrowed to a single model (Nemotron 3 Ultra) after MiMo free access was revoked in late May 2026. The shipped note "Free tier available" unders…
- **`glhf`** — The shipped freeNote ("Free tier for open-source model inference") is now stale. The free beta ended in January 2025; GLHF Chat is now a paid pay-as-you-go service. There is no ongoing recurring free…
- **`groq`** — The shipped freeNote "30 RPM / 14.4K RPD" is accurate only for llama-3.1-8b-instant. Most other models (including llama-3.3-70b-versatile) have a much lower 1K RPD cap. The note omits model-specific …
- **`hackclub`** — The "30+ models" count appears accurate and still matches. The core offering remains free for Hack Club members. No evidence of tightening — still "$0 ALWAYS FREE" per the homepage. The freeNote omit…
- **`huggingchat`** — The shipped freeNote ("Free LLM chat — no subscription required. Rate limits apply.") is partially accurate but significantly understates the restrictions. The free tier now operates on a hard $0.10/…
- **`huggingface`** — Significantly tightened. The shipped freeNote ("Free Inference API for thousands of models") implied unlimited/generous free access, but as of mid-2025 the free tier is capped at $0.10/month in recur…
- **`hyperbolic`** — Our shipped freeNote says "$1-5 trial credits on signup" — the $1 trial credit portion is accurate, but the "$5" figure refers to the minimum deposit required to unlock GPU rental (not free credits g…
- **`iflytek`** — Catalog says "Free Spark Lite models" — this is broadly accurate. However the current reality is more nuanced: only Spark Lite is free (the Max 100M token offer was a one-time promo, not recurring); …
- **`inclusionai`** — Our shipped freeNote says "Free Ling-2.6-flash model (262K context)" without specifying token limits. Reality is more specific: the free tier is 500K tokens/day (shared across all models), with a 2 Q…
- **`inference-net`** — The shipped freeNote states "$25 free credits on signup plus research grants." The current pricing page shows only $1 recurring monthly credits with no mention of a $25 signup bonus or research grant…
- **`jina-reader`** — Our shipped freeNote was "(none)", which is incorrect. Jina Reader has had a publicly documented free tier since launch: keyless access at 20 RPM plus a 10M one-time token grant with a free API key. …
- **`kiro`** — Catalog shipped freeNote "(none)" — but Kiro has a documented, perpetual free tier of 50 credits/month. The free tier existed since Kiro's public launch (pricing formalized ~October 2025). This is a …
- **`kluster`** — The $5 free credits on signup appears to still match. However, there is evidence of an additional permanent free tier (post-credit) with undocumented limits, which may represent an improvement over t…
- **`llm7`** — Rate limits have increased from the shipped freeNote (20 RPM / 100 req/hr → 40 RPM / 200 req/hr). The "no signup required" claim is now outdated — a free token from token.llm7.io is now required (tho…
- **`longcat`** — Significant change from shipped freeNote of "(none)": the platform actively provides 5M free tokens/day on a recurring daily basis to all API users during the public beta.
- **`mistral`** — The shipped freeNote ("Free Experiment tier: rate-limited access to all models") is directionally correct but understated. Current reality adds specific documented limits: 2 RPM, 500K TPM, 1B tokens/…
- **`monsterapi`** — The shipped freeNote says "Free credits for decentralized GPU inference" which is partially accurate — there are one-time trial credits on signup. However, the recurring free tier has 0 credits/month…
- **`morph`** — The shipped freeNote mentions "250K credits/month" which matches the current credit allocation; however, the more significant constraint is 200 requests/month which was not captured in the original c…
- **`muse-spark-web`** — The shipped freeNote ("Free with login — Meta AI platform with Llama models") is broadly accurate regarding login requirement and Llama model access. No tightening of the free tier was detected; it r…
- **`nlpcloud`** — The shipped freeNote says "Trial credits for new accounts," which implies a one-time trial. In reality, NLP Cloud's free tier is a recurring monthly free plan (10,000 requests/month), not trial credi…
- **`nomic`** — Our shipped freeNote says "Free Nomic Embed API" with no qualification, implying ongoing free access. Reality is a one-time 1M-token trial credit only — after that token budget is consumed, paid subs…
- **`nous-research`** — The shipped freeNote ("Free tier: 50 RPM, 500,000 TPM") does not match the current Nous Portal product. The portal launched April 27, 2026 and structures its free tier as $0.10/month in recurring cre…
- **`nvidia`** — The "40 RPM, 70+ models" rate limit element matches the catalog, but the freeNote framing as a simple dev-access tier undersells that the old one-time credit pool has been removed — access is now tru…
- **`ollama-cloud`** — Our shipped freeNote is "(none)" — this is stale. Ollama Cloud launched a cloud inference product with a genuine free tier that provides light weekly GPU-time-based access to hosted open models.
- **`openrouter`** — RPD tightened from 200 to 50 for zero-credit accounts (RPM unchanged at 20). The catalog note was accurate on RPM but overstated the RPD by 4x for the no-credits baseline tier.
- **`phind`** — Our shipped freeNote describes an active free chat/code-search service — but Phind is fully discontinued as of January 16, 2026. The catalog entry should be marked discontinued and removed from activ…
- **`pollinations`** — Partially matches — the "no API key required" claim is still true for anonymous access, but the catalog freeNote omits that: (1) rate limits do apply (interval throttle of ~1 req/6-15s for anonymous …
- **`predibase`** — The shipped freeNote ($25 free trial credits, 30-day validity) still matches current documentation. However, the catalog omits the concurrent 20,000 tokens/day serverless rate limit that applies duri…
- **`publicai`** — The shipped freeNote ("Free community inference tier") is broadly accurate but understates the specificity: the 20 RPM rate limit is now documented. No major tightening found; the service remains fre…
- **`puter`** — Partially matches: the "500+ models" count is still accurate. However "users pay via Puter account" understates the reality — free accounts receive an undocumented starting credit that can be exhaust…
- **`qoder`** — Our catalog ships freeNote "(none)", but Qoder does have a free tier: a Community Edition with unlimited basic-model completions (daily-capped, unspecified limit) plus a one-time 14-day/300-credit Pr…
- **`qwen-web`** — The shipped freeNote ("Free — Qwen models via chat.qwen.ai with login token") is now stale. The login-token/OAuth free API path was terminated on 2026-04-15. The qwen-web executor will receive 401 er…
- **`sambanova`** — Our shipped note only described the one-time $5 credit (30-day validity). The current reality includes a permanent recurring free tier with documented rate limits (20 RPM, 20 RPD, 200k TPD) that pers…
- **`sensenova`** — Our shipped freeNote says "Free SenseTime models" which is vague but directionally correct — free access does exist. However, reality is more nuanced: free access is a time-limited public beta (Token…
- **`serper-search`** — The shipped freeNote says "(none)" which is partially accurate — there is no recurring free plan — but Serper does offer 2,500 one-time trial credits on signup. The catalog note could be more precise…
- **`siliconflow`** — Partially matches but more nuanced: the $1 free credits are a one-time trial (not recurring), while the "permanently free models" component still holds — free $0 models continue to exist (Qwen3-8B, D…
- **`sparkdesk`** — Partially matches — the shipped freeNote "Free iFlytek Spark models" is accurate in that Spark Lite is permanently free, but understates the constraint (2 QPS per App ID) and overstates scope (only S…
- **`stepfun`** — The shipped freeNote "Free Step-2 models" is stale. Step-2 LLM free access is no longer offered; the platform has transitioned to Step 3.x models on a paid-per-token basis with no free LLM tier. Only…
- **`t3-web`** — The shipped freeNote is broadly accurate (limited model access, Pro unlocks 50+ models for $8/month), but misses two key updates: (1) the free tier now resets daily instead of monthly (changed around…
- **`tavily-search`** — Catalog ships freeNote "(none)" implying no free tier, but Tavily does in fact offer a documented recurring free tier of 1,000 credits/month with no credit card required. This is a significant discre…
- **`tencent`** — Largely matches — the shipped freeNote ("Free Hunyuan Lite models") is accurate. Hunyuan-lite has been permanently free since May 2024 and remains so as of 2026. The catalog note undersells the detai…
- **`theoldllm`** — Our shipped freeNote was "(none)" — this still matches in the sense that no structured API/free tier offering exists; the service remains a UI-only chat wrapper with no catalogable API tier.
- **`together`** — The shipped note says "$25 signup credits + 3 permanently free models" but reality shows far more permanently free models (~80, not 3). The $25 trial credit figure is contested — official billing doc…
- **`uncloseai`** — Largely matches — still free forever with no signup. However, the ToS (terms-of-use.html) clarifies IP-based throttling exists for excessive use and prohibits building competing ML services without a…
- **`veoaifree-web`** — The shipped freeNote states "6 requests/hour" but no such explicit limit is currently documented anywhere on veoaifree.com. The site claims unlimited free generation with no login. The models listed …
- **`voyage-ai`** — The shipped freeNote "200M free tokens for embeddings and reranking" is directionally correct on the token count but misleading — it omits that this is a one-time per-account allocation, not a recurr…
- **`yi`** — The shipped freeNote "Free Yi-Light models" references a model name ("Yi-Light") that does not appear in any current 01.AI documentation or model catalog — no such model is listed on platform.01.ai, …
---
## Glossary
| Term | Meaning |
| ---------- | -------------------------------------------------------- |
| **RPM** | Requests per minute |
| **RPD** | Requests per day |
| **RPH** | Requests per hour |
| **RPS** | Requests per second |
| **TPM** | Tokens per minute |
| **TPD** | Tokens per day |
| **Neuron** | Cloudflare's compute unit (~1 output token) |
| **LKGP** | Last-known-good provider — auto-combo strategy |
| **P2C** | Power-of-two choices — auto-combo load balancer strategy |
| Term | Meaning |
|---|---|
| **RPM / RPD / RPH** | Requests per minute / day / hour |
| **TPM / TPD** | Tokens per minute / day |
| **Documented grant** | Provider publishes an explicit daily/monthly token cap (defensible budget) |
| **Theoretical ceiling** | `rate-limit × 24/7 × 30d` — a maximum, not a granted budget |
| **Neuron** | Cloudflare compute unit (~1 output token) |
> Generated from per-provider research on 2026-06-05. Re-run the research workflow (see `_tasks/features-v3.8.12`) to refresh.

19
Guardrails.md

@@ -264,3 +264,22 @@ exercise the full flow without DB or network access.
forced-bridge model list
- `docs/architecture/RESILIENCE_GUIDE.md` — orthogonal layer (circuit breaker, cooldowns)
- `docs/reference/ENVIRONMENT.md` — full env var reference
## Injection-guard route coverage & red-team (Fase 8 · Bloco D)
O injection-guard (`createInjectionGuard` / `withInjectionGuard`) cobre todas as rotas
que aceitam prompt do usuário. Respeita `INJECTION_GUARD_MODE` (default `warn` = só loga;
`block` = retorna HTTP 400 `SECURITY_001`).
| Tipo | Rotas | Modo default |
|---|---|---|
| Texto (já existente) | `/v1/chat/completions`, `/v1/completions`, `/v1/relay/chat/completions` | warn |
| Generativas | `/v1/messages`, `/v1/responses`, `/v1/images/generations`, `/v1/images/edits`, `/v1/videos/generations`, `/v1/music/generations`, `/v1/audio/speech` | warn |
| Dados | `/v1/embeddings`, `/v1/rerank`, `/v1/search`, `/v1/moderations` | warn |
A extração de texto (`extractMessageContents`) cobre `messages`/`input`/`prompt`/`query`+`documents`/`instructions`/`system`.
**Red-team (nightly, `nightly-llm-security.yml`):** promptfoo valida que cada rota bloqueia
o corpus OWASP-LLM em `INJECTION_GUARD_MODE=block`; garak roda probes (skip sem secret).
`moderations` é incluída por consistência — operadores em block-mode podem isentá-la via
`resolveDisabledGuardrails`.

1
Kiro-Setup.md

@@ -1,5 +1,6 @@
> 🌍 [View in other languages](Languages)
# Kiro Setup Guide
This guide covers adding Kiro (AWS-hosted AI coding assistant) accounts to OmniRoute,

96
MCP-Server.md

@@ -3,13 +3,13 @@
# OmniRoute MCP Server Documentation
> Model Context Protocol server with 87 tools across routing, cache, compression, memory, skills, and proxy operations.
> Model Context Protocol server with 87 tools across routing, cache, compression, memory, skills, proxy, and context source operations.
>
> Source of truth: `open-sse/mcp-server/schemas/tools.ts` (30 tools) + `open-sse/mcp-server/tools/memoryTools.ts` (3 tools) + `open-sse/mcp-server/tools/skillTools.ts` (4 tools). Tool registration and scope wiring lives in `open-sse/mcp-server/server.ts`.
> Source of truth: `open-sse/mcp-server/schemas/tools.ts` (30 tools) + `open-sse/mcp-server/tools/memoryTools.ts` (3 tools) + `open-sse/mcp-server/tools/skillTools.ts` (4 tools) + `open-sse/mcp-server/tools/notionTools.ts` (6 tools). Tool registration and scope wiring lives in `open-sse/mcp-server/server.ts`.
![MCP tool inventory (87 tools by category)](../diagrams/exported/mcp-tools-37.svg)
![MCP tool inventory (43 tools by category)](../diagrams/exported/mcp-tools-43.svg)
> Source: [diagrams/mcp-tools-37.mmd](../diagrams/mcp-tools-37.mmd)
> Source: [diagrams/mcp-tools-43.mmd](../diagrams/mcp-tools-43.mmd) (regenerate via `npm run docs:render-diagrams`).
## Installation
@@ -155,27 +155,67 @@ the runtime compression model behind these tools.
Defined in `open-sse/mcp-server/tools/memoryTools.ts`. Auth/scope is enforced through the standard MCP scope pipeline.
| Tool | Description |
| :------------------------ | :---------------------------------------------------------------------------------- |
| `omniroute_memory_search` | Search memories by query / type / API key with token-budget enforcement |
| `omniroute_memory_add` | Add a new memory entry (`factual` / `episodic` / `procedural` / `semantic`) |
| `omniroute_memory_clear` | Clear memories for an API key, optionally filtered by type or `olderThan` timestamp |
| Tool | Scopes | Description |
| :------------------------ | :--------------- | :---------------------------------------------------------------------------------- |
| `omniroute_memory_search` | `read:memory` | Search memories by query / type / API key with token-budget enforcement |
| `omniroute_memory_add` | `write:memory` | Add a new memory entry (`factual` / `episodic` / `procedural` / `semantic`) |
| `omniroute_memory_clear` | `write:memory` | Clear memories for an API key, optionally filtered by type or `olderThan` timestamp |
## Skill Tools (4)
Defined in `open-sse/mcp-server/tools/skillTools.ts`. Backed by `src/lib/skills/registry` + `src/lib/skills/executor`.
| Tool | Description |
| :---------------------------- | :-------------------------------------------------------------------------------- |
| `omniroute_skills_list` | List registered skills with optional filtering by API key, name, or enabled state |
| `omniroute_skills_enable` | Enable or disable a specific skill by ID |
| `omniroute_skills_execute` | Execute a skill with provided input and return the execution record |
| `omniroute_skills_executions` | List recent skill execution history |
| Tool | Scopes | Description |
| :---------------------------- | :-------------- | :-------------------------------------------------------------------------------- |
| `omniroute_skills_list` | `read:skills` | List registered skills with optional filtering by API key, name, or enabled state |
| `omniroute_skills_enable` | `write:skills` | Enable or disable a specific skill by ID |
| `omniroute_skills_execute` | `execute:skills`| Execute a skill with provided input and return the execution record |
| `omniroute_skills_executions` | `read:skills` | List recent skill execution history |
## Notion Context Source (6)
Defined in `open-sse/mcp-server/tools/notionTools.ts`. Token stored in `key_value` table via `src/lib/db/notion.ts`. REST client in `src/lib/notion/api.ts`. Settings API in `src/app/api/settings/notion/route.ts`. Dashboard UI in `src/app/(dashboard)/dashboard/endpoint/components/NotionSourceCard.tsx`.
Configure your Notion integration token from the **Context Sources** tab in the Endpoint dashboard, or via the REST API:
```bash
# Set token
curl -X POST http://localhost:20128/api/settings/notion \
-H "Content-Type: application/json" \
-d '{"token": "ntn_..."}'
# Check status
curl http://localhost:20128/api/settings/notion
# Disconnect
curl -X DELETE http://localhost:20128/api/settings/notion
```
| Tool | Scopes | Description |
| :--------------------------- | :--------------- | :------------------------------------------------------------------------------------ |
| `omniroute_notion_search` | `read:notion` | Full-text search across all pages and databases |
| `omniroute_notion_list_databases` | `read:notion` | List all accessible databases with schema metadata |
| `omniroute_notion_get_database` | `read:notion` | Get database schema by ID |
| `omniroute_notion_query_database` | `read:notion` | Query a database with filters, sorts, and pagination |
| `omniroute_notion_read` | `read:notion` | Read a page or block by ID with its content |
| `omniroute_notion_append_blocks` | `write:notion`| Append children blocks to a parent block (max 100 per request) |
## Agent Skill Catalog Tools (3)
Defined in `open-sse/mcp-server/tools/agentSkillTools.ts`. Backed by `src/lib/agentSkills/catalog`. These tools expose the 42-entry Agent Skills documentation catalog to MCP clients and external agents. Scope: `read:catalog`.
| Tool | Scopes | Description |
| :--------------------------------- | :------------- | :--------------------------------------------------------------------------------------------------------------- |
| `omniroute_agent_skills_list` | `read:catalog` | List all 42 agent skills with optional `category` (api\|cli) and `area` filters; returns metadata + coverage |
| `omniroute_agent_skills_get` | `read:catalog` | Get full metadata + SKILL.md content for a single skill by canonical `id` |
| `omniroute_agent_skills_coverage` | `read:catalog` | Coverage stats: how many of the 22 API and 20 CLI skills have SKILL.md files on the filesystem vs catalog totals |
See [AGENT-SKILLS.md](./AGENT-SKILLS.md) for the full catalog and how external agents consume it.
## Related Frameworks (v3.8.0)
The MCP tool inventory above (87 tools = 30 base + 3 memory + 4 skills) is intentionally
scoped to runtime routing/cache/compression/memory/skills/proxy operations. Two adjacent
The MCP tool inventory above (43 tools = 30 core + 3 memory + 4 skills + 6 notion) is intentionally
scoped to runtime routing/cache/compression/memory/skills/proxy/context-source operations. Two adjacent
frameworks ship alongside the MCP server in v3.8.0 and are documented separately:
### Cloud Agents
@@ -244,11 +284,17 @@ MCP tools are authenticated through API key scopes. Scope enforcement is central
| `read:compression` | `compression_status`, `list_compression_combos`, `compression_combo_stats` |
| `write:compression` | `compression_configure`, `set_compression_engine` |
| `read:proxies` | `oneproxy_fetch`, `oneproxy_rotate`, `oneproxy_stats` |
| `read:notion` | `notion_search`, `notion_list_databases`, `notion_get_database`, `notion_query_database`, `notion_read` |
| `write:notion` | `notion_append_blocks` |
| `read:memory` | `memory_search` |
| `write:memory` | `memory_add`, `memory_clear` |
| `read:skills` | `skills_list`, `skills_executions` |
| `write:skills` | `skills_enable` |
| `execute:skills` | `skills_execute` |
| `read:catalog` | `agent_skills_list`, `agent_skills_get`, `agent_skills_coverage` |
Wildcard scopes are supported: `read:*` grants all read-scopes, `*` grants full access.
Memory and Skill tools currently do not declare static scope requirements in their definitions; access is gated by the caller's API key and audited through the standard MCP audit pipeline.
---
## Environment Variables
@@ -291,7 +337,7 @@ The heartbeat snapshot contains:
"transport": "stdio",
"scopesEnforced": false,
"allowedScopes": [],
"toolCount": 37
"toolCount": 43
}
```
@@ -325,9 +371,19 @@ Use the dashboard or the `/api/mcp/audit` and `/api/mcp/audit/stats` REST endpoi
| `open-sse/mcp-server/tools/compressionTools.ts` | Compression tool handlers |
| `open-sse/mcp-server/tools/memoryTools.ts` | Memory tool definitions (3 tools) |
| `open-sse/mcp-server/tools/skillTools.ts` | Skill tool definitions (4 tools) |
| `open-sse/mcp-server/tools/notionTools.ts` | Notion context source tool definitions (6 tools) |
| `open-sse/mcp-server/tools/gamificationTools.ts`| Gamification tool definitions (8 tools) |
| `open-sse/mcp-server/tools/pluginTools.ts` | Plugin registration and management tools (8 tools) |
| `src/app/api/mcp/status/route.ts` | `/api/mcp/status` endpoint |
| `src/app/api/mcp/tools/route.ts` | `/api/mcp/tools` endpoint |
| `src/app/api/mcp/sse/route.ts` | `/api/mcp/sse` SSE transport route |
| `src/app/api/mcp/stream/route.ts` | `/api/mcp/stream` Streamable HTTP transport route |
| `src/app/api/mcp/audit/route.ts` | `/api/mcp/audit` audit log query |
| `src/app/api/mcp/audit/stats/route.ts` | `/api/mcp/audit/stats` aggregated audit metrics |
| `src/lib/notion/api.ts` | Notion REST API client (retry, timeout, error classification) |
| `src/lib/db/notion.ts` | Notion token persistence (`key_value` table) |
| `src/app/api/settings/notion/route.ts` | Notion settings API (GET/POST/DELETE) |
| `src/app/(dashboard)/dashboard/endpoint/components/NotionSourceCard.tsx` | Notion token management UI |
| `tests/unit/notion-api.test.ts` | Notion API client tests (7) |
| `tests/unit/notion-tools.test.ts` | Notion tools scope enforcement tests (10) |
| `tests/unit/db/notion.test.mjs` | Notion DB module tests (3) |

567
Memory.md

@@ -4,7 +4,7 @@
# Memory System
> **Source of truth:** `src/lib/memory/` and `src/app/api/memory/`
> **Last updated:** 2026-05-13 — v3.8.0
> **Last updated:** 2026-05-28 — v3.8.6 (plan 21 — Memory Engine Redesign)
OmniRoute provides persistent conversational memory keyed by API key (and
optionally session id). Memories are extracted automatically from LLM responses
@@ -24,17 +24,143 @@ Client → /v1/chat/completions (apiKeyInfo resolved upstream)
→ resolveMemoryOwnerId(apiKeyInfo) # extracts id
→ getMemorySettings() # cached settings
→ shouldInjectMemory(body, {enabled}) # gate
→ retrieveMemories(apiKeyId, config) # SQL + optional FTS5
→ retrieveMemories(apiKeyId, config) # SQL + FTS5 + optional vector
→ injectMemory(body, memories, provider) # system or user message
→ upstream provider call
→ on response: extractFacts(text, apiKeyId, sessionId) # non-blocking
→ setImmediate → createMemory(fact) per match
→ embed(content) + upsertVector(id, vec)
```
The injection and extraction call-sites are wired in
`open-sse/handlers/chatCore.ts` (look for `retrieveMemories`, `injectMemory`,
and `extractFacts`).
## Engine architecture (3-tier resolution)
The Memory Engine resolves the retrieval path at runtime based on available
infrastructure and settings. Three tiers exist, applied in priority order:
```
┌─────────────────────────────────────────────────────────────┐
│ TIER 0 — Keyword (FTS5) │
│ Always available. SQLite FTS5 full-text search over │
│ content + key. Used when strategy = "exact" or as fallback. │
└──────────────────────────────────┬──────────────────────────┘
│ strategy = semantic|hybrid?
▼
┌─────────────────────────────────────────────────────────────┐
│ TIER 1 — Embedded Vector (sqlite-vec) │
│ sqlite-vec v0.1.9 loaded via db.loadExtension(). │
│ KNN brute-force over Float32 vectors. Active when: │
│ • sqlite-vec loadExtension succeeds │
│ • An embedding source is available (remote | static | │
│ transformers) that can produce a Float32Array │
│ • vec_memories table exists (created on first ready()) │
└──────────────────────────────────┬──────────────────────────┘
│ qdrant.enabled?
▼
┌─────────────────────────────────────────────────────────────┐
│ TIER 2 — Qdrant (opt-in external vector database) │
│ When enabled, replaces sqlite-vec for semantic/hybrid. │
│ Requires running Qdrant instance + configured host/port. │
└─────────────────────────────────────────────────────────────┘
```
Degradation is automatic and transparent:
- If sqlite-vec fails to load, tier 1 is unavailable → falls back to tier 0.
- If embedding source returns an error, tier 1 falls back to tier 0.
- If Qdrant is unhealthy, tier 2 falls back to tier 1 (or tier 0 if tier 1
is also unavailable).
## Embedding sources
The embedding layer (`src/lib/memory/embedding/`) resolves which source to use
based on `MemorySettingsExtended.embeddingSource`:
| Source | Description | Key required | Cold start |
| -------------- | ------------------------------------------------------------------------------- | ------------ | ---------- |
| `remote` | Uses a configured provider's embedding API (OpenAI, Cohere, etc.) | Yes | None |
| `static` | Local lookup-table embedding via `potion-base-8M` (WordPiece + mean pooling) | No | ~200ms |
| `transformers` | Local ONNX inference via `@huggingface/transformers` v4, `all-MiniLM-L6-v2` | No | ~3s + ~400MB RAM |
| `auto` | Runtime resolution: remote (if key exists) → static → transformers → null | Depends | Depends |
**Resolution order for `auto`:**
1. Find first provider in `listEmbeddingProviders()` with `hasKey === true` → `remote`.
2. If `settings.staticEnabled === true` → `static`.
3. If `settings.transformersEnabled === true` → `transformers`.
4. Otherwise → `null` (degrades to FTS5 keyword search).
The embedding cache (`src/lib/memory/embedding/cache.ts`) uses an in-memory
LRU map keyed by `${source}:${model}:${dim}:${sha256(text)}`, capped at
`MEMORY_EMBEDDING_CACHE_MAX` entries (default 1000) with a TTL of
`MEMORY_EMBEDDING_CACHE_TTL_MS` (default 5 min). Shared across all callers
per process lifecycle.
## Hybrid RRF (k=60)
When `strategy = "hybrid"` and the vector store is available, retrieval uses
Reciprocal Rank Fusion to merge FTS5 and vector results:
```
RRF(d) = Σ 1 / (k + rank_i(d)) where k = 60 (configurable via MEMORY_RRF_K)
i
```
Concretely:
1. Run FTS5 search → ranked list `R_fts` (position 1..N).
2. Run KNN vector search → ranked list `R_vec` (position 1..M).
3. For each unique `memoryId`:
`rrf_score = 1/(60 + fts_rank)` + `1/(60 + vec_rank)` (0 if not in list).
4. Sort by `rrf_score` DESC, apply token budget walk.
RRF is well-known to be effective without needing score normalization across
heterogeneous retrieval systems. The default `k=60` is from the original
Cormack et al. paper and works well for small corpora (<10k memories).
## Backfill (lazy + reindex)
When the embedding model changes (detected via `embedding_signature`), the
vector store is rebuilt and all existing memories are marked
`needs_reindex = 1` in the `memories` table.
**Lazy backfill**: On the next retrieval, any memory missing a vector entry is
embedded and inserted into `vec_memories` before the search runs. This
amortizes the backfill cost across real requests without blocking startup.
**Explicit reindex**: The Engine tab in `/dashboard/memory` provides a
"Reindex Now" button that calls `POST /api/memory/reindex`. The handler calls
`runReindexBatch()` from `src/lib/memory/reindex.ts`, which processes up to
`limit` pending entries per request. Progress can be polled via
`GET /api/memory/engine-status` (`vectorStore.needsReindex`).
The `memory_vec_meta` table (migration `073_memory_vec.sql`) stores:
- `active_dim` — current vector dimension (null = not yet calibrated).
- `embedding_signature` — `${source}:${model}:${dim}` used to detect changes.
- `last_reset_at` — timestamp of last full reset.
- `vec_loaded` — 0/1 flag whether sqlite-vec loaded successfully.
## Settings extension
Seven new fields were added to `MemorySettingsExtended` (plan 21, D9) in
`src/shared/schemas/memory.ts`, persisted via `src/lib/db/settings.ts`:
| Field | Type | Default | Description |
| ---------------------- | --------------------------------------------- | ------------ | --------------------------------------------- |
| `embeddingSource` | `"remote" \| "static" \| "transformers" \| "auto"` | `"auto"` | Which embedding source to use |
| `embeddingProviderModel` | `string \| null` | `null` | Provider/model in `provider/model` format |
| `transformersEnabled` | `boolean` | `false` | Opt-in for Transformers.js (MiniLM, ~400MB) |
| `staticEnabled` | `boolean` | `false` | Opt-in for static potion-base-8M local model |
| `rerankEnabled` | `boolean` | `false` | Enable reranking step (adds +200-500ms/req) |
| `rerankProviderModel` | `string \| null` | `null` | Rerank provider/model in `provider/model` format |
| `vectorStore` | `"sqlite-vec" \| "qdrant" \| "auto"` | `"auto"` | Which vector backend to use |
These are exposed via `GET /PUT /api/settings/memory` (schema `MemorySettingsExtendedSchema`).
> **TODO (D20):** Scope `global` (sharing memories across all API keys) is not
> implemented in this release. It requires schema changes and a global retrieval
> path. Track separately.
## Storage Layers
### Primary: SQLite (`memories` table)
@@ -75,10 +201,10 @@ Used by `retrieval.ts` for the `semantic` and `hybrid` strategies (see below).
The retrieval code guards with `hasTable("memory_fts")` and falls back to
chronological order if the FTS table is missing or the FTS query throws.
### Optional: Qdrant (vector store)
### Optional: Qdrant (vector store tier 2)
`src/lib/memory/qdrant.ts` implements an optional Qdrant integration for true
semantic memory:
`src/lib/memory/qdrant.ts` implements an optional Qdrant integration as tier 2
vector store. Enabled via `qdrantEnabled` in settings / toggle in Engine tab.
- `upsertSemanticMemoryPoint()` — embed `key + content` with the configured
embedding model, ensure the collection exists (creates cosine-distance
@@ -87,24 +213,24 @@ apiKeyId, sessionId, key, content, metadata, createdAtUnix, expiresAtUnix}`.
- `searchSemanticMemory(query, topK, scope)` — embed the query, search the
collection filtered by `kind = "omniroute_memory"` and optionally by
`apiKeyId` / `sessionId`. Caps `topK` to `[1, 20]`.
- `deleteSemanticMemoryPoint(id)` — single point delete.
- `deleteSemanticMemoryPoint(id)` — single point delete. Called by
`deleteMemory()` after the SQLite row is removed (D15).
- `cleanupSemanticMemoryPoints({retentionDays})` — bulk delete points whose
`expiresAtUnix` is in the past or whose `createdAtUnix` is older than the
retention cutoff. Counts first so the dashboard can show actual numbers.
- `checkQdrantHealth()` — `GET /readyz` health probe with latency.
> **TODO**: The chat pipeline (`chatCore.ts`) and the in-tree `retrieveMemories()`
> implementation do not currently call `upsertSemanticMemoryPoint` or
> `searchSemanticMemory`. The Qdrant integration is feature-flagged via
> `qdrantEnabled` in settings, but at the time of writing the
> `searchSemanticMemory` results are not fused into retrieval — the
> `semantic`/`hybrid` retrieval strategies use SQLite FTS5 only. The settings UI
> in `dashboard/settings → MemorySkillsTab` exposes Qdrant config, health,
> search test, and cleanup, but the corresponding `/api/settings/qdrant`,
> `/api/settings/qdrant/health`, `/api/settings/qdrant/search`, and
> `/api/settings/qdrant/cleanup` routes are referenced from the UI but **not
> present** under `src/app/api/settings/qdrant/` (only `embedding-models/` is
> wired). Treat Qdrant as preview/optional plumbing.
The settings UI exposes Qdrant config, health check, semantic search test,
and cleanup in the **Engine tab** of `/dashboard/memory`. The corresponding
routes under `src/app/api/settings/qdrant/` are all wired as of v3.8.6:
| Route | Method | Description |
| ----- | ------ | ----------- |
| `/api/settings/qdrant` | `GET` / `PUT` | Read / update Qdrant settings |
| `/api/settings/qdrant/health` | `GET` | Liveness probe + latency |
| `/api/settings/qdrant/search` | `POST` | Semantic search test |
| `/api/settings/qdrant/cleanup` | `POST` | Remove expired / old points |
| `/api/settings/qdrant/embedding-models` | `GET` | List available embedding models |
## Memory Types
@@ -199,6 +325,8 @@ Memory configuration is **stored in the DB settings table**, not in env vars.
in-process; `invalidateMemorySettingsCache()` is called by the settings PUT
route after writes.
### Legacy fields (all versions)
| DB key | Type | Default | UI control |
| --------------------- | ------- | -------------------------------------------------- | ----------------------------------------------- |
| `memoryEnabled` | boolean | `true` | Memory on/off |
@@ -210,14 +338,38 @@ route after writes.
Note: the UI strategy `"recent"` maps to the internal `"exact"` retrieval
strategy via `toMemoryRetrievalConfig()` (chronological order).
### New fields (v3.8.6, plan 21 D9)
See also the "Settings extension" section above for field descriptions.
| DB key | API field | Default |
| ------------------------- | ---------------------- | ------------- |
| `memoryEmbeddingSource` | `embeddingSource` | `"auto"` |
| `memoryEmbeddingModel` | `embeddingProviderModel` | `null` |
| `memoryTransformersEnabled` | `transformersEnabled` | `false` |
| `memoryStaticEnabled` | `staticEnabled` | `false` |
| `memoryRerankEnabled` | `rerankEnabled` | `false` |
| `memoryRerankModel` | `rerankProviderModel` | `null` |
| `memoryVectorStore` | `vectorStore` | `"auto"` |
Qdrant-related DB keys (`qdrantEnabled`, `qdrantHost`, `qdrantPort`,
`qdrantApiKey`, `qdrantCollection` default `"omniroute_memory"`,
`qdrantEmbeddingModel` default `"openai/text-embedding-3-small"`) are read by
`normalizeQdrantConfig()` in `qdrant.ts`.
No `MEMORY_*` or `QDRANT_*` env vars exist today — everything is per-instance
DB settings. `OMNIROUTE_MEMORY_MB` (commented out in `.env.example`) is
unrelated and refers to Node heap sizing.
### Environment variables (v3.8.6)
Six optional env vars tune the engine's runtime behaviour (documented in `.env.example`):
| Variable | Default | Description |
| ------------------------------- | ------- | -------------------------------------------------- |
| `MEMORY_EMBEDDING_CACHE_TTL_MS` | `300000` | Embedding cache TTL (5 min) |
| `MEMORY_EMBEDDING_CACHE_MAX` | `1000` | Max entries in embedding LRU cache |
| `MEMORY_TRANSFORMERS_MODEL` | `Xenova/all-MiniLM-L6-v2` | HF repo for Transformers.js model |
| `MEMORY_STATIC_MODEL` | `minishlab/potion-base-8M` | HF repo for static potion model |
| `MEMORY_STATIC_CACHE_DIR` | `<DATA_DIR>/embeddings` | Where to store downloaded models |
| `MEMORY_VEC_TOP_K` | `20` | Default top-K for vector search |
| `MEMORY_RRF_K` | `60` | RRF k constant for hybrid search |
## Summarisation (`summarization.ts`)
@@ -237,15 +389,39 @@ loss is one-way: original text is overwritten.
All endpoints require management auth (`requireManagementAuth`).
| Method | Path | Description |
| -------- | ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `GET` | `/api/memory` | Paginated list with filters: `apiKeyId`, `type`, `sessionId`, `q`, `limit`, `page`, `offset`. Response includes `stats.total` and `stats.byType` |
| `POST` | `/api/memory` | Create entry (Zod-validated: `content`, `key`, optional `type`, `sessionId`, `apiKeyId`, `metadata`, `expiresAt`). Calls `createMemory()` which upserts on `(apiKeyId, key)` |
| `GET` | `/api/memory/[id]` | Fetch a single entry by UUID |
| `DELETE` | `/api/memory/[id]` | Delete an entry; returns 404 when missing |
| `GET` | `/api/memory/health` | Runs `verifyExtractionPipeline("health-check")` — round-trip create→list→delete to confirm the store is alive. Returns `{working, latencyMs, error?}` |
| `GET` | `/api/settings/memory` | Current normalised `MemorySettings` |
| `PUT` | `/api/settings/memory` | Update one or more of `enabled`, `maxTokens`, `retentionDays`, `strategy`, `skillsEnabled` |
### Core memory endpoints (existing + updated)
| Method | Path | Description |
| ---------- | --------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `GET` | `/api/memory` | Paginated list with filters: `apiKeyId`, `type`, `sessionId`, `q`, `limit`, `page`, `offset`. Response includes `stats.total`, `stats.tokensUsed`, `stats.hitRate`, `cacheStats` |
| `POST` | `/api/memory` | Create entry (Zod-validated: `content`, `key`, optional `type`, `sessionId`, `apiKeyId`, `metadata`, `expiresAt`). Calls `createMemory()` which upserts on `(apiKeyId, key)` |
| `GET` | `/api/memory/[id]` | Fetch a single entry by UUID |
| `PUT` | `/api/memory/[id]` | Update entry fields (`type`, `key`, `content`, `metadata`). Body: `MemoryUpdatePutSchema`. Also syncs vector if embedding source available. |
| `DELETE` | `/api/memory/[id]` | Delete an entry; also deletes from `vec_memories` (D15) and Qdrant best-effort. Returns 404 when missing. |
| `GET` | `/api/memory/health` | Runs `verifyExtractionPipeline("health-check")` — round-trip create→list→delete. Returns `{working, latencyMs, error?}` |
### New memory engine endpoints (plan 21)
| Method | Path | Description |
| -------- | ---------------------------------- | --------------------------------------------------------------------------------------- |
| `POST` | `/api/memory/retrieve-preview` | Dry-run of `retrieveMemories` — returns ranked results with score, tier, tokens. Body: `RetrievePreviewSchema`. Does NOT inject or modify memories. |
| `GET` | `/api/memory/embedding-providers` | Lists providers with embedding models, indicating which have a configured API key. |
| `GET` | `/api/memory/engine-status` | Returns full engine status: keyword tier, embedding resolution, vector store stats, Qdrant health, rerank config. Shape: `MemoryEngineStatusSchema`. |
| `POST` | `/api/memory/summarize` | Manually trigger memory compaction. Body: `MemorySummarizeSchema` (`olderThanDays`, `apiKeyId?`, `dryRun`). Returns `{candidates, tokensSaved}`. |
| `POST` | `/api/memory/reindex` | Trigger vector reindex for memories with `needs_reindex=1`. Body: `MemoryReindexSchema` (`force`). Returns `{started, pending}`. |
### Settings endpoints
| Method | Path | Description |
| -------- | ---------------------------------- | ------------------------------------------------------------------------ |
| `GET` | `/api/settings/memory` | Current normalised `MemorySettingsExtended` (7 new fields + legacy) |
| `PUT` | `/api/settings/memory` | Update any field from `MemorySettingsExtendedSchema` (12 total fields) |
| `GET` | `/api/settings/qdrant` | Current Qdrant settings (`QdrantSettingsSchema`) |
| `PUT` | `/api/settings/qdrant` | Update Qdrant settings. Body: `QdrantSettingsUpdateSchema`. `apiKey` = empty string removes key. |
| `GET` | `/api/settings/qdrant/health` | Liveness probe against configured Qdrant instance. Returns `QdrantHealthResultSchema`. |
| `POST` | `/api/settings/qdrant/search` | Semantic search test against Qdrant. Body: `QdrantSearchSchema` (`query`, `topK`). |
| `POST` | `/api/settings/qdrant/cleanup` | Remove Qdrant points for expired / old memories. |
| `GET` | `/api/settings/qdrant/embedding-models` | List embedding models available for Qdrant. |
The `/api/memory` list query supports either `page`-based pagination
(`parsePaginationParams`) **or** raw `offset` — when `offset` is present it
@@ -256,31 +432,55 @@ takes precedence and a derived `page` is computed for the response shape.
When the MCP server is enabled, three memory tools are registered:
- `omniroute_memory_search` — `{apiKeyId, query?, type?, maxTokens?, limit?}`
→ wraps `retrieveMemories()` with `retrievalStrategy: "exact"`, optionally
filters by `type`, and reports `totalTokens`.
→ wraps `retrieveMemories()`. As of v3.8.6 (D16), the `strategy` is read
from `getMemorySettings()` instead of being hardcoded to `"exact"`. If
`query` is provided and `strategy` is `semantic` or `hybrid`, the vector
store is used when available.
- `omniroute_memory_add` — `{apiKeyId, sessionId?, type, key, content,
metadata?}` → wraps `createMemory()`.
metadata?}` → wraps `createMemory()`. Accepts only the 4 canonical types:
`factual`, `episodic`, `procedural`, `semantic` (D17).
- `omniroute_memory_clear` — `{apiKeyId, type?, olderThan?}` → lists matching
entries, optionally filters by created-before timestamp, then deletes each
via `deleteMemory()`.
via `deleteMemory()` (which also removes vectors from sqlite-vec + Qdrant).
See [MCP-SERVER.md](./MCP-SERVER.md) for transport and scope details.
## Dashboard
## Dashboard (Memory Studio)
`src/app/(dashboard)/dashboard/memory/page.tsx` provides:
`src/app/(dashboard)/dashboard/memory/page.tsx` is now a **3-tab Studio**:
### Tab: Memórias / Memories
- Concept card (collapsible "How it works" explainer).
- Real-time list, search, and pagination (debounced 300 ms).
- Type filter (`factual` / `episodic` / `procedural` / `semantic` / all).
- Add-memory modal (key, content, type).
- Delete per row.
- Inline edit (pencil button → `PUT /api/memory/[id]`).
- Delete per row (with confirmation dialog).
- JSON export of the current page; JSON import via file picker.
- Stat cards: `totalEntries`, `tokensUsed`, `hitRate`.
- "Compact old" button → `POST /api/memory/summarize` (dry-run first shows
candidate count, then confirms).
- A green/red health dot driven by `GET /api/memory/health`.
- Stat cards: `totalEntries`, `tokensUsed`, `hitRate` (the latter two come
from the API stats payload).
Memory and Qdrant settings live under
`/dashboard/settings → Memory & Skills` (`MemorySkillsTab.tsx`).
### Tab: Playground
- Query input + strategy selector (Exact / Semantic / Hybrid) + token budget.
- "Simulate" → `POST /api/memory/retrieve-preview` — shows ranked results with
`score`, `tier`, `tokens`, `vecScore`, `ftsScore`.
- Resolution panel showing which embedding source / vector store was used and
whether a fallback occurred.
### Tab: Engine
- Engine status panel (keyword FTS5 chip, embedding chip, vector store chip,
Qdrant health chip, rerank chip).
- "Reindex Now" button → `POST /api/memory/reindex`.
- Embedding source selector (auto / remote / static / transformers + toggles).
- Qdrant config card (enable toggle, host/port/collection/key, test connection,
semantic search test, cleanup).
- Rerank config card (enable toggle, provider/model selector).
Memory and Qdrant settings also live under
`/dashboard/settings → Memory & Skills` (`MemorySkillsTab.tsx`) for
the legacy/global settings surface.
## Caching
@@ -313,12 +513,289 @@ default TTL 5 min).
- [API_REFERENCE.md](../reference/API_REFERENCE.md) — broader API surface.
- Source modules:
- `src/lib/memory/types.ts`, `schemas.ts`
- `src/lib/memory/store.ts`, `retrieval.ts`, `injection.ts`
- `src/lib/memory/store.ts`, `retrieval.ts`, `injection.ts`, `reindex.ts`
- `src/lib/memory/extraction.ts`, `summarization.ts`, `verify.ts`
- `src/lib/memory/settings.ts`, `qdrant.ts`, `cache.ts`
- `src/lib/memory/vectorStore.ts` — sqlite-vec + hybrid RRF
- `src/lib/memory/embedding/index.ts` — multi-source embedding layer
- `src/lib/memory/embedding/types.ts`, `remote.ts`, `staticPotion.ts`,
`transformersLocal.ts`, `cache.ts`
- `src/shared/schemas/memory.ts` — Zod schemas for all memory API bodies
- `src/shared/schemas/qdrant.ts` — Zod schemas for Qdrant settings/ops
- `src/lib/db/memoryVec.ts` — CRUD for `memory_vec_meta`
- `src/lib/db/migrations/015_create_memories.sql`,
`022_add_memory_fts5.sql`, `023_fix_memory_fts_uuid.sql`
`022_add_memory_fts5.sql`, `023_fix_memory_fts_uuid.sql`,
`073_memory_vec.sql`
- `src/app/api/memory/route.ts`, `[id]/route.ts`, `health/route.ts`
- `src/app/api/memory/retrieve-preview/route.ts`
- `src/app/api/memory/engine-status/route.ts`
- `src/app/api/memory/embedding-providers/route.ts`
- `src/app/api/memory/summarize/route.ts`
- `src/app/api/memory/reindex/route.ts`
- `src/app/api/settings/memory/route.ts`
- `open-sse/handlers/chatCore.ts` (injection / extraction wiring)
- `open-sse/mcp-server/tools/memoryTools.ts`
- `src/app/api/settings/qdrant/route.ts` + sub-routes
- `src/app/(dashboard)/dashboard/memory/` — Studio UI (page + components +
tabs + hooks)
- `open-sse/handlers/chatCore.ts` (injection / extraction wiring)
- `open-sse/mcp-server/tools/memoryTools.ts`
---
## Choosing an Embedding Provider (v3.8.16+)
OmniRoute's memory engine supports **four embedding sources** (`src/lib/memory/embedding/`). Each has different trade-offs in **latency, cost, model quality, and setup complexity**.
### The Four Providers
| Provider | Source | Latency | Cost | Quality | Setup |
|----------|--------|---------|------|---------|-------|
| `transformers` | Local ONNX model (Xenova/all-MiniLM-L6-v2) | ~50-150ms (CPU) | Free | Good | `npm install` only |
| `static` | Pre-computed vectors (cached) | <1ms | Free | N/A (depends on cache hit) | None |
| `remote` | OpenAI / Cohere / Voyage API | ~100-300ms | $0.02-0.10/1M tokens | Excellent | API key |
| `cache` | In-memory LRU layer over any source | <1ms (hit), full latency (miss) | Free | Same as underlying | None |
### Decision Tree
```
What's your deployment context?
│
┌───────────┼───────────┬──────────────┐
│ │ │ │
DEV/TEST SMALL PROD LARGE PROD EDGE / OFFLINE
│ │ │ │
▼ ▼ ▼ ▼
transformers transformers remote (Qdrant) transformers
(free, no API) (best quality) (no internet)
│ │ │ │
└────────┬──┴───────────┴──────────────┘
│
▼
ALWAYS add `cache` layer on top
(LruCache wraps any provider)
```
### Database & API Configuration
Memory embedding options are configured via the Settings API/UI, not environment variables. The relevant settings database keys under Settings (`normalizeMemorySettings` in `src/lib/memory/settings.ts`) are:
- `memoryEmbeddingSource`: `"transformers"` (local), `"remote"` (API-based, e.g. OpenAI), `"static"` (external store), or `"auto"`
- `memoryEmbeddingProviderModel`: Model identifier for remote/static sources (e.g., `"text-embedding-3-small"`)
- `memoryTransformersEnabled`: `true` | `false`
- `memoryStaticEnabled`: `true` | `false`
- `memoryVectorStore`: `"sqlite-vec"`, `"qdrant"`, or `"auto"`
#### Local Model (`transformers`)
Uses transformers.js internally to run local models:
```bash
# Env vars read in code (src/lib/memory/embedding/index.ts):
MEMORY_TRANSFORMERS_MODEL=Xenova/all-MiniLM-L6-v2 # HF model repo
MEMORY_STATIC_MODEL=minishlab/potion-base-8M # HF static potion model
MEMORY_STATIC_CACHE_DIR=<DATA_DIR>/embeddings # Cache directory
```
#### LRU Embedding Cache
The cache is always on by default and configured via env vars:
```bash
MEMORY_EMBEDDING_CACHE_MAX=1000 # Max cached items
MEMORY_EMBEDDING_CACHE_TTL_MS=300000 # TTL (5 min)
```
### Performance Numbers
Benchmark on a typical 4-core x86 server (texts ~100 tokens each):
| Provider | p50 | p95 | p99 | Cost / 1M embeddings |
|----------|-----|-----|-----|-----------------------|
| `transformers` (CPU) | 80ms | 180ms | 350ms | Free |
| `remote` (OpenAI) | 120ms | 220ms | 400ms | ~$0.02 (ada-002) / $0.13 (3-large) |
| `static` (Qdrant) | 15ms | 30ms | 60ms | Depends on Qdrant hosting |
| `cache` (hit) | <1ms | <1ms | 2ms | Free |
---
## Fact Extraction Patterns (v3.8.16+)
The `extraction.ts` module (`src/lib/memory/extraction.ts`) uses **regex pattern matching** to extract structured facts from conversation messages. Understanding these patterns helps you tune extraction quality for your use case.
### Default Pattern Categories
| Category | Example pattern | Captures |
|----------|-----------------|----------|
| PREFERENCE_PATTERNS | `"I prefer <X>"`, `"I like <X>"`, `"I hate <X>"` | User preferences |
| DECISION_PATTERNS | `"I'll use <X>"`, `"I decided to <X>"`, `"I went with <X>"` | User decisions (episodic) |
| PATTERN_PATTERNS | `"I usually <X>"`, `"I always <X>"`, `"I never <X>"` | Persistent behavioral patterns |
### Example Patterns (Simplified)
```ts
// From src/lib/memory/extraction.ts
const PREFERENCE_PATTERNS = [
/\bI\s+(?:really\s+)?prefer\s+([^.,\n]+)/gi,
/\bI\s+(?:really\s+)?like\s+([^.,\n]+)/gi,
/\bI\s+(?:hate|dislike|avoid)\s+([^.,\n]+)/gi
];
const DECISION_PATTERNS = [
/\bI'?(?:ll|will)\s+use\s+([^.,\n]+)/gi,
/\bI\s+(?:have\s+)?decided\s+(?:to\s+)?([^.,\n]+)/gi
];
const PATTERN_PATTERNS = [
/\bI\s+usually\s+([^.,\n]+)/gi,
/\bI\s+always\s+([^.,\n]+)/gi
];
```
### What Gets Extracted
When a user says:
> "I prefer TypeScript. I'll use Postgres for this project. I always commit before pushing. I don't like Python."
Extraction produces 4 memories:
| Key | Category | Type | Content |
|-----|----------|------|---------|
| `preference:typescript` | preference | factual | "TypeScript" |
| `decision:postgres_for_this_project` | decision | episodic | "Postgres for this project" |
| `pattern:commit_before_pushing` | pattern | factual | "commit before pushing" |
| `preference:python` | preference | factual | "Python" |
### Extraction Limits
To prevent runaway extraction, the following limits apply:
| Min content length | 3 chars |
| Max content length | 500 chars |
### When to Disable Extraction
Extraction runs automatically whenever memory is enabled; there is no separate
extraction-only toggle. To turn it off, disable memory entirely (`enabled: false`
via `PUT /api/settings/memory`). Consider doing so when:
- You have high message volume and the extraction cost is non-trivial
- Your conversations are mostly transient (chat, debugging) with no long-term value
- You're already capturing context via custom plugins
---
## Hybrid RRF Tuning (v3.8.16+)
The **Reciprocal Rank Fusion (RRF)** algorithm combines FTS5 (keyword) and vector (semantic) results. The `k` parameter controls how much weight is given to lower-ranked results.
### The Formula
For each candidate memory, the RRF score is:
```
RRF(d) = Σ 1 / (k + rank_i(d))
```
Where:
- `k` is the constant (default 60)
- `rank_i(d)` is the rank of document `d` in the i-th retrieval system (FTS, vector)
- The sum runs over all retrieval systems
### How `k` Affects Results
| `k` value | Effect | Best for |
|-----------|--------|----------|
| `k=0` | Pure rank fusion (no smoothing) | Theoretical baseline |
| `k=10-30` | Heavily weights top results, low-rank barely contributes | When top-3 results are usually correct |
| **`k=60`** (default) | Balanced — top-10 results all contribute meaningfully | General-purpose retrieval |
| `k=100+` | Flatter — even low-rank results can dominate if they appear in multiple systems | When recall > precision is critical |
### Tuning `k` in Practice
```bash
# Default
MEMORY_RRF_K=60
# Aggressive precision (small memory, few docs)
MEMORY_RRF_K=20
# Maximum recall (large memory, varied queries)
MEMORY_RRF_K=120
```
**Example with `k=20`:**
- FTS rank 1 → contribution `1/21 = 0.048`
- FTS rank 10 → contribution `1/30 = 0.033`
- Vector rank 1 → contribution `0.048`
- Combined max: `0.096`
**Example with `k=60`:**
- FTS rank 1 → contribution `1/61 = 0.016`
- FTS rank 10 → contribution `1/70 = 0.014`
- Vector rank 1 → contribution `0.016`
- Combined max: `0.033`
With higher `k`, the **relative difference** between top-1 and rank-10 is smaller, so the algorithm relies more on **consensus across retrieval systems** than on top-rank confidence.
### When to Change `k`
| Symptom | Try |
|---------|-----|
| Top result always wins, but it's wrong | **Lower** k (e.g., 20) — top-rank confidence matters more |
| Right answer is in top-5 but not top-1 | **Higher** k (e.g., 100) — flatter scoring rewards consensus |
| Recall is high but precision is low | **Lower** k — sharpen the ranking |
| Recall is low (missing relevant docs) | **Higher** k — give lower-ranked docs a chance |
### RRF Weighting
The reciprocal rank fusion uses equal weights for semantic vector rank and full-text search rank:
```
RRF(d) = 1/(k + rank_vector) + 1/(k + rank_fts)
```
There are no environment variables to adjust individual weights (`MEMORY_RRF_VECTOR_WEIGHT`/`MEMORY_RRF_FTS_WEIGHT` do not exist).
---
## Summarization Strategy (v3.8.16+)
The `summarization.ts` module (`src/lib/memory/summarization.ts`) compresses older memories to keep the active set small while preserving recall.
### When Summarization Triggers
| Trigger | Threshold (default) |
|---------|---------------------|
| Manual trigger via API | n/a |
### What Gets Summarized
Two entry points are exported from `summarization.ts`:
- **`summarizeMemories(apiKeyId, sessionId?, maxTokens = 4000)`** — condenses the
memories for a session into a single summary text bounded by a token budget.
- **`summarizeMemoriesOlderThan(apiKeyId, days, dryRun)`** — the age-based
compaction used by the API: it selects every memory older than `days`, builds
one condensed summary memory from them, and (when `dryRun` is `false`) deletes
the originals. Pass `dryRun: true` to preview the candidate set and token total
without modifying anything.
There is no tag/key clustering pass or per-memory "core vs summarizable" scoring —
selection is purely the age cutoff, and the summary text is a condensed,
type-prefixed line per candidate.
### Triggering Summarization
Summarization is **manual / opt-in** — the `autoSummarize` setting is `false` by
default, so nothing is compacted automatically. Trigger it via the API:
```bash
curl -X POST http://localhost:20128/api/memory/summarize \
-H "Authorization: Bearer $OMNIROUTE_KEY"
```
To leave it off, simply keep `autoSummarize` at its default (`false`).
### Summarization Quality Tips
- **Preview first with `dryRun`** — `summarizeMemoriesOlderThan(..., true)` returns
the candidate list and total token count so you can confirm what would be merged
before deleting the originals.
- **Run summarization during low-traffic hours** if you have a large memory corpus — the LLM call is the slow part
```bash
# Cron-style: summarize at 3am daily
0 3 * * * curl -X POST http://localhost:20128/api/memory/summarize \
-H "Authorization: Bearer $OMNIROUTE_KEY"
```

4
OmniRoute-vs-Alternatives.md

@@ -16,7 +16,7 @@ Objective feature comparison vs popular open-source AI routers.
| **Auto-fallback combos** | **15 strategies** | priority-based | tier-based | weighted |
| **Tier 1/2/3 fallback (subscription→cheap→free)** | ✅ + UI | manual | n/a | manual |
| **Token compression** | RTK (47 filters) + Caveman | none | none | none |
| **Built-in MCP server** | ✅ 87 tools, 13 scopes | ❌ | ❌ | ❌ |
| **Built-in MCP server** | ✅ 87 tools, 30 scopes | ❌ | ❌ | ❌ |
| **A2A protocol** | ✅ 5 skills | ❌ | ❌ | ❌ |
| **Memory (FTS5 + vector)** | ✅ | ❌ | ❌ | ❌ |
| **Guardrails (PII, injection, vision)** | ✅ | partial | ❌ | ✅ paid |
@@ -28,7 +28,7 @@ Objective feature comparison vs popular open-source AI routers.
| **CLI with system tray (no Electron)** | ✅ | ❌ | n/a | n/a |
| **CLI machine-ID auto-auth** | ✅ | ❌ | n/a | n/a |
| **Dashboard** | Next.js 16 | basic | proprietary | proprietary |
| **i18n** | **40+ locales** | ❌ | ❌ | ⚠ |
| **i18n** | **42+ locales** | ❌ | ❌ | ⚠ |
| **Public agent skills (SKILL.md)** | ✅ 10 | ❌ | ❌ | ❌ |
| **Tunnel support (Cloudflared, Tailscale, Ngrok)** | ✅ | ❌ | n/a | n/a |
| **License** | MIT | MIT | proprietary | proprietary |

393
Provider-Reference.md

@@ -5,7 +5,7 @@
> **Auto-generated** from `src/shared/constants/providers.ts` — do not edit by hand.
> Regenerate with: `npm run gen:provider-reference`
> **Last generated:** 2026-05-17
> **Last generated:** 2026-06-15
Total providers: **226**. See category breakdown below.
@@ -28,185 +28,229 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
---
## Free Tier (OAuth-first or no-key) (5)
## OAuth Providers (19)
| ID | Alias | Name | Tags | Website | Notes |
| ------------ | ------------ | ---------- | ---- | ------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `amazon-q` | `aq` | Amazon Q | Free | [link](https://aws.amazon.com/q/developer/) | Uses the same AWS Builder ID or imported refresh-token flow as Kiro, but keeps Amazon Q connections separate. |
| `gemini-cli` | `gemini-cli` | Gemini CLI | Free | — | Uses Gemini CLI OAuth / Cloud Code credentials. Pro models require an eligible Google account or paid plan. |
| `kiro` | `kr` | Kiro AI | Free | — | — |
| `qoder` | `if` | Qoder AI | Free | — | — |
| `qwen` | `qw` | Qwen Code | Free | — | ⚠️ **DEPRECATED.** Qwen OAuth free tier was discontinued on 2026-04-15. Use 'bailian-coding-plan', 'alibaba', 'alibaba-cn', or 'openrouter' provider with API key instead. |
| ID | Alias | Name | Tags | Website | Notes |
| ------------- | ------------ | -------------------- | ----- | ------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `agy` | `agy` | Antigravity CLI | OAuth | [link](https://antigravity.google) | Import your Antigravity CLI (`agy`) login (paste/upload its token file), auto-detect a local CLI login, or sign in with Google. Shares the Antigravity backend (incl. Claude models). |
| `amazon-q` | `aq` | Amazon Q | OAuth | [link](https://aws.amazon.com/q/developer/) | Uses the same AWS Builder ID or imported refresh-token flow as Kiro, but keeps Amazon Q connections separate. |
| `antigravity` | — | Antigravity | OAuth | — | — |
| `claude` | `cc` | Claude Code | OAuth | — | — |
| `cline` | `cl` | Cline | OAuth | — | — |
| `codex` | `cx` | OpenAI Codex | OAuth | — | — |
| `cursor` | `cu` | Cursor IDE | OAuth | — | — |
| `devin-cli` | `dv` | Devin CLI (Official) | OAuth | [link](https://cli.devin.ai) | Requires the Devin CLI binary. Run `devin auth login` to authenticate, or provide your WINDSURF_API_KEY. Install: https://cli.devin.ai |
| `gemini-cli` | `gemini-cli` | Gemini CLI | OAuth | — | Uses Gemini CLI OAuth / Cloud Code credentials. Pro models require an eligible Google account or paid plan. |
| `github` | `gh` | GitHub Copilot | OAuth | — | — |
| `gitlab-duo` | `gitlab-duo` | GitLab Duo | OAuth | [link](https://docs.gitlab.com/user/duo_agent_platform/code_suggestions/) | OAuth application with ai_features + read_user scopes. Configure GITLAB_DUO_OAUTH_CLIENT_ID and optionally GITLAB_DUO_OAUTH_CLIENT_SECRET on this OmniRoute instance. |
| `kilocode` | `kc` | Kilo Code | OAuth | — | — |
| `kimi-coding` | `kmc` | Kimi Coding | OAuth | — | — |
| `kiro` | `kr` | Kiro AI | OAuth | — | Free tier: 50 credits/month (~25K–100K tokens). ⚠️ Kiro ToS prohibits third-party proxy/harness use. |
| `qoder` | `if` | Qoder AI | OAuth | — | — |
| `qwen` | `qw` | Qwen Code | OAuth | — | ⚠️ **DEPRECATED.** Qwen OAuth free tier was discontinued on 2026-04-15. Use 'bailian-coding-plan', 'alibaba', 'alibaba-cn', or 'openrouter' provider with API key instead. |
| `trae` | `tr` | Trae | OAuth | [link](https://trae.ai) | Trae is an AI-native IDE by ByteDance (SOLO remote agent). Authorize via trae.ai in the popup, or sign in at solo.trae.ai and paste the Cloud-IDE-JWT (sent as 'Authorization: Cloud-IDE-JWT <token>', ~14-day lifetime) as the access token; web_id/biz_user_id/user_unique_id/scope/tenant/region propagate via providerSpecificData. No headless refresh for pasted tokens — re-paste on expiry. |
| `windsurf` | `ws` | Windsurf (Devin CLI) | OAuth | [link](https://windsurf.com) | In the Windsurf / VS Code IDE, open the command palette and run `Windsurf: Provide Auth Token` (or click the Jupyter "Get Windsurf Authentication Token" button), then copy the shown token and paste it here. Note: opening windsurf.com/show-auth-token directly only renders a "Redirecting" page — the IDE must initiate the flow (it adds a `?state=...` param) for the token to appear. |
| `zed` | `zd` | Zed IDE | OAuth | [link](https://zed.dev) | Zed stores LLM provider credentials (OpenAI, Anthropic, Google, Mistral, xAI) in the OS keychain. Use the Import button below to discover and import them automatically. |
## OAuth Providers (11)
## Web Cookie Providers (22)
| ID | Alias | Name | Tags | Website | Notes |
| ------------- | ------------ | -------------------- | ----- | ------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `antigravity` | — | Antigravity | OAuth | — | — |
| `claude` | `cc` | Claude Code | OAuth | — | — |
| `cline` | `cl` | Cline | OAuth | — | — |
| `codex` | `cx` | OpenAI Codex | OAuth | — | — |
| `cursor` | `cu` | Cursor IDE | OAuth | — | — |
| `devin-cli` | `dv` | Devin CLI (Official) | OAuth | [link](https://cli.devin.ai) | Requires the Devin CLI binary. Run `devin auth login` to authenticate, or provide your WINDSURF_API_KEY. Install: https://cli.devin.ai |
| `github` | `gh` | GitHub Copilot | OAuth | — | — |
| `gitlab-duo` | `gitlab-duo` | GitLab Duo | OAuth | [link](https://docs.gitlab.com/user/duo_agent_platform/code_suggestions/) | OAuth application with ai_features + read_user scopes. Configure GITLAB_DUO_OAUTH_CLIENT_ID and optionally GITLAB_DUO_OAUTH_CLIENT_SECRET on this OmniRoute instance. |
| `kilocode` | `kc` | Kilo Code | OAuth | — | — |
| `kimi-coding` | `kmc` | Kimi Coding | OAuth | — | — |
| `windsurf` | `ws` | Windsurf (Devin CLI) | OAuth | [link](https://windsurf.com) | Sign in at windsurf.com to get your token. Visit windsurf.com/show-auth-token after logging in and paste it here, or use the device-code login flow. |
| ID | Alias | Name | Tags | Website | Notes |
| ----------------- | ------------- | ---------------------------- | ---------- | -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `adapta-web` | `adp-web` | Adapta.org (Adapta One Web) | Web cookie | [link](https://agent.adapta.one) | Paste your \_\_client cookie value from .clerk.agent.adapta.one (DevTools → Application → Cookies) |
| `blackbox-web` | `bb-web` | Blackbox Web (Subscription) | Web cookie | [link](https://app.blackbox.ai) | Paste your \_\_Secure-authjs.session-token value or full cookie header from app.blackbox.ai |
| `chatgpt-web` | `cgpt-web` | ChatGPT Web (Plus/Pro) | Web cookie | [link](https://chatgpt.com) | Paste your \_\_Secure-next-auth.session-token cookie value from chatgpt.com |
| `claude-web` | `cw` | Claude Web | Web cookie | [link](https://claude.ai) | Paste your session cookie from claude.ai |
| `copilot-web` | `copilot` | Microsoft Copilot Web | Web cookie | [link](https://copilot.microsoft.com) | Paste your access_token from copilot.microsoft.com (or export a .har file from DevTools while logged in) |
| `deepseek-web` | `ds-web` | DeepSeek Web | Web cookie | [link](https://chat.deepseek.com) | Paste your userToken from chat.deepseek.com — DevTools → Application → Local Storage → userToken |
| `doubao-web` | `db` | Doubao Web (ByteDance) | Web cookie | [link](https://www.doubao.com) | Paste your session cookie from doubao.com (DevTools → Application → Cookies) |
| `gemini-business` | `gembiz` | Gemini Business (Enterprise) | Web cookie | [link](https://business.gemini.google) | From your enterprise account: open business.gemini.google/home/cid/{your-cid}, then copy **Secure-1PSID and **Secure-1PSIDTS cookies from DevTools → Application → Cookies. Paste as a cookie header below. |
| `gemini-web` | `gweb` | Gemini Web (Free) | Web cookie | [link](https://gemini.google.com) | Paste your **Secure-1PSID cookie value from gemini.google.com. Optionally add **Secure-1PSIDTS separated by semicolon. |
| `grok-web` | `gw` | Grok Web (Subscription) | Web cookie | [link](https://grok.com) | Paste the full grok.com cookie line from DevTools → Application → Cookies. Include both `sso` and `sso-rw` (e.g. `sso=...; sso-rw=...`) — Grok's anti-bot rejects `sso` on its own. |
| `huggingchat` | `huggingchat` | HuggingChat (Free) | Web cookie | [link](https://huggingface.co/chat) | Paste your hf-chat cookie value from huggingface.co/chat (DevTools → Application → Cookies → hf-chat). Optional — works without auth for basic use. |
| `inner-ai` | `in-ai` | Inner.ai (Subscription) | Web cookie | [link](https://app.innerai.com) | Paste your token cookie and email separated by a space: open DevTools → Application → Cookies → .innerai.com, copy the token value, then append a space and your Inner.ai login email. Example: eyJhbG... user@example.com |
| `kimi-web` | `kimi-web` | Kimi Web (Moonshot AI) | Web cookie | [link](https://kimi.moonshot.cn) | Paste your session cookie from kimi.moonshot.cn (DevTools → Application → Cookies) |
| `lmarena` | `lma` | LMArena (Free) | Web cookie | [link](https://lmarena.ai) | Paste your session cookie from lmarena.ai (DevTools → Application → Cookies). Optional — works with free tier for basic comparisons. |
| `muse-spark-web` | `ms-web` | Muse Spark Web (Meta AI) | Web cookie | [link](https://www.meta.ai) | Paste your abra_sess value or full cookie header from meta.ai |
| `perplexity-web` | `pplx-web` | Perplexity Web (Pro/Max) | Web cookie | [link](https://www.perplexity.ai) | Paste your \_\_Secure-next-auth.session-token cookie value from perplexity.ai |
| `phind` | `ph` | Phind (Free) | Web cookie | [link](https://www.phind.com) | Paste your session cookie from phind.com (DevTools → Application → Cookies). Optional — works with free tier. |
| `poe-web` | `poe` | Poe Web (Subscription) | Web cookie | [link](https://poe.com) | Paste your p-b cookie value from poe.com (DevTools → Application → Cookies → p-b) |
| `qwen-web` | `qwen-web` | Qwen Web (Free) | Web cookie | [link](https://chat.qwen.ai) | Open chat.qwen.ai, log in, then open DevTools → Application → Local Storage → copy the "token" value (or use tongyi_sso_ticket cookie as Bearer token). |
| `t3-web` | `t3chat` | t3.chat (Pro/Free) | Web cookie | [link](https://t3.chat) | Open t3.chat in your browser, log in, then open DevTools → Application → Local Storage → https://t3.chat. Copy the value of 'convex-session-id'. Also open DevTools → Network, copy the Cookie header from any request. Paste both values here. See provider setup docs for a step-by-step guide. |
| `v0-vercel-web` | `v0` | v0 Vercel Web (Code Gen) | Web cookie | [link](https://v0.dev) | Paste your session cookie from v0.dev (DevTools → Application → Cookies) |
| `venice-web` | `ven` | Venice Web (Privacy) | Web cookie | [link](https://venice.ai) | Paste your session cookie from venice.ai (DevTools → Application → Cookies) |
## Web Cookie Providers (7)
## API Key Providers (paid / paid-with-free-credits) (152)
| ID | Alias | Name | Tags | Website | Notes |
| ---------------- | ---------- | --------------------------- | ---------- | --------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `blackbox-web` | `bb-web` | Blackbox Web (Subscription) | Web cookie | [link](https://app.blackbox.ai) | Paste your \_\_Secure-authjs.session-token value or full cookie header from app.blackbox.ai |
| `chatgpt-web` | `cgpt-web` | ChatGPT Web (Plus/Pro) | Web cookie | [link](https://chatgpt.com) | Paste your \_\_Secure-next-auth.session-token cookie value from chatgpt.com |
| `deepseek-web` | `ds-web` | DeepSeek Web | Web cookie | [link](https://chat.deepseek.com) | Paste your ds_session_id cookie from chat.deepseek.com |
| `grok-web` | `gw` | Grok Web (Subscription) | Web cookie | [link](https://grok.com) | Paste your sso= cookie value from grok.com |
| `muse-spark-web` | `ms-web` | Muse Spark Web (Meta AI) | Web cookie | [link](https://www.meta.ai) | Paste your abra_sess value or full cookie header from meta.ai |
| `perplexity-web` | `pplx-web` | Perplexity Web (Pro/Max) | Web cookie | [link](https://www.perplexity.ai) | Paste your \_\_Secure-next-auth.session-token cookie value from perplexity.ai |
| `t3-web` | `t3chat` | t3.chat (Pro/Free) | Web cookie | [link](https://t3.chat) | Pro: $8/mo, 50+ models. Free tier: limited models. Requires Cookie header + convex-session-id from DevTools. **Skeleton — endpoint URL not yet confirmed (TODO post-devtools-capture).** |
| ID | Alias | Name | Tags | Website | Notes |
| --------------------- | -------------- | ------------------------------- | --------------------- | -------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `360ai` | `360ai` | 360 AI | API key | [link](https://ai.360.cn) | Get API key at ai.360.cn |
| `agentrouter` | `agentrouter` | AgentRouter | API key, aggregator | [link](https://agentrouter.org) | $200 free credits on signup - multi-model routing gateway |
| `ai21` | `ai21` | AI21 Labs | API key | [link](https://www.ai21.com) | $10 trial credits on signup (valid 3 months), no credit card required |
| `aimlapi` | `aiml` | AI/ML API | API key, aggregator | [link](https://aimlapi.com) | $0.025/day free credits — 200+ models (GPT-4o, Claude, Gemini, Llama) via single endpoint |
| `alibaba` | `ali` | Alibaba | API key | [link](https://dashscope-intl.aliyuncs.com) | — |
| `alibaba-cn` | `ali-cn` | Alibaba (China) | API key | [link](https://dashscope.aliyuncs.com) | — |
| `anthropic` | `anthropic` | Anthropic | API key | [link](https://platform.claude.com) | — |
| `api-airforce` | `af` | Api.airforce | API key | [link](https://api.airforce) | 55 free tier models including Grok-3, Claude 3.7, Qwen3, Kimi-K2, Gemini 2.5 Flash, DeepSeek-V3 |
| `arcee-ai` | `arcee` | Arcee AI | API key | [link](https://arcee.ai) | Get API key at arcee.ai |
| `azure-ai` | `azure-ai` | Azure AI Foundry | API key, enterprise | [link](https://learn.microsoft.com/azure/ai-foundry) | Use your Azure AI Foundry key. Base URL can be https://<resource>.services.ai.azure.com/openai/v1/ or https://<resource>.openai.azure.com/openai/v1/. |
| `azure-openai` | `azure` | Azure OpenAI | API key, enterprise | [link](https://azure.microsoft.com/products/ai-services/openai-service) | Use your Azure OpenAI API key. Base URL should be your resource endpoint, for example https://my-resource.openai.azure.com. |
| `baichuan` | `baichuan` | Baichuan | API key | [link](https://baichuan.com) | Get API key at platform.baichuan-ai.com |
| `baidu` | `baidu` | Baidu (ERNIE) | API key | [link](https://yiyan.baidu.com) | Get API key at console.bce.baidu.com |
| `bailian-coding-plan` | `bcp` | Alibaba Coding Plan | API key | [link](https://www.alibabacloud.com/help/en/model-studio/coding-plan) | — |
| `baseten` | `baseten` | Baseten | API key | [link](https://baseten.co) | $30 free trial credits for GPU inference |
| `bazaarlink` | `bzl` | BazaarLink | API key | [link](https://bazaarlink.ai) | Free tier with auto:free routing — zero-cost inference, no credit card required |
| `bedrock` | `bedrock` | Amazon Bedrock | API key, enterprise | [link](https://aws.amazon.com/bedrock) | Use your Amazon Bedrock API key and configure the AWS region where your models are enabled (for example eu-west-2). OmniRoute calls Bedrock's native Converse API directly. |
| `black-forest-labs` | `bfl` | Black Forest Labs | API key, image | [link](https://blackforestlabs.ai) | — |
| `blackbox` | `bb` | Blackbox AI | API key | [link](https://blackbox.ai) | Free tier: unlimited basic chat plus Minimax-M2.5, no credit card required |
| `bluesminds` | `bm` | BluesMinds | API key | [link](https://www.bluesminds.com) | Free daily pi credits — supports 200+ models including GPT-4o, GPT-4.1, Claude Sonnet 4.5, Gemini 2.0 Flash, DeepSeek V4, Qwen, Kimi K2 |
| `byteplus` | `bpm` | BytePlus ModelArk | API key | [link](https://console.byteplus.com/ark) | — |
| `bytez` | `bytez` | Bytez | API key | [link](https://bytez.com) | $1 free credits, refreshes every 4 weeks |
| `cablyai` | `cablyai` | CablyAI | API key, aggregator | [link](https://cablyai.com) | Bearer API key for the CablyAI OpenAI-compatible gateway. |
| `cerebras` | `cerebras` | Cerebras | API key | [link](https://inference.cerebras.ai) | Free Trial: 1M tokens/day, 30K TPM, 5 RPM — no credit card. |
| `chutes` | `chutes` | Chutes.ai | API key, aggregator | [link](https://chutes.ai) | Bearer API key for the Chutes OpenAI-compatible gateway. |
| `clarifai` | `clarifai` | Clarifai | API key, enterprise | [link](https://docs.clarifai.com) | Use your Clarifai PAT or app-specific API key. OmniRoute targets the OpenAI-compatible endpoint at https://api.clarifai.com/v2/ext/openai/v1 and authenticates with Authorization: Key <token>. |
| `cloudflare-ai` | `cf` | Cloudflare Workers AI | API key | [link](https://developers.cloudflare.com/workers-ai) | Requires API Token AND Account ID (found at dash.cloudflare.com) |
| `codestral` | `codestral` | Codestral | API key | [link](https://mistral.ai) | — |
| `cohere` | `cohere` | Cohere | API key | [link](https://cohere.com) | Free Trial: 1,000 API calls/month for testing, no credit card required |
| `command-code` | `cmd` | Command Code | API key | [link](https://commandcode.ai/) | Use a Command Code API key. Requests are sent to Command Code's /alpha/generate endpoint. |
| `coze` | `coze` | Coze | API key | [link](https://coze.com) | Get API key at coze.com/open/api |
| `crof` | `crof` | CrofAI | API key | [link](https://crof.ai) | — |
| `databricks` | `databricks` | Databricks | API key, enterprise | [link](https://www.databricks.com) | — |
| `datarobot` | `datarobot` | DataRobot | API key, enterprise | [link](https://docs.datarobot.com) | Use your DataRobot API token. Optional Base URL can be the account root (for LLM Gateway) or a deployment URL under /api/v2/deployments/<id>. |
| `deepinfra` | `deepinfra` | DeepInfra | API key | [link](https://deepinfra.com) | Free signup credits for API testing and model exploration |
| `deepseek` | `ds` | DeepSeek | API key | [link](https://platform.deepseek.com) | 5M free tokens on signup - no credit card required |
| `dify` | `dify` | Dify | API key | [link](https://dify.ai) | Get API key from your Dify instance. |
| `doubao` | `doubao` | Doubao | API key | [link](https://doubao.com) | Get API key at console.volcengine.com |
| `empower` | `empower` | Empower | API key, aggregator | [link](https://docs.empower.dev) | Bearer API key for the Empower OpenAI-compatible endpoint. |
| `fal-ai` | `fal` | Fal.ai | API key, image | [link](https://fal.ai) | — |
| `featherless-ai` | `featherless` | Featherless AI | API key | [link](https://featherless.ai) | Free tier available — no credit card required |
| `fenayai` | `fenayai` | FenayAI | API key, aggregator | [link](https://fenayai.com) | Bearer API key for the FenayAI OpenAI-compatible gateway. |
| `firecrawl` | `fc` | Firecrawl | API key | [link](https://firecrawl.dev) | — |
| `fireworks` | `fireworks` | Fireworks AI | API key | [link](https://fireworks.ai) | $1 free starter credits on signup for API testing |
| `freeaiapikey` | `faik` | FreeAIAPIKey | API key | [link](https://freeaiapikey.com) | — |
| `freemodel-dev` | `fmd` | FreeModel.dev | API key | [link](https://freemodel.dev) | $300 free credits on signup — no credit card required. Access GPT-5.4 and GPT-5.5 (OpenAI's latest flagship models) through an OpenAI-compatible API. |
| `friendliai` | `friendli` | FriendliAI | API key | [link](https://friendli.ai) | Free tier for serverless inference — no credit card required |
| `galadriel` | `galadriel` | Galadriel | API key | [link](https://galadriel.com) | — |
| `gemini` | `gemini` | Gemini (Google AI Studio) | API key | [link](https://aistudio.google.com) | Free forever: 1,500 req/day for Gemini 2.5 Flash — no credit card, get key at aistudio.google.com |
| `getgoapi` | `ggo` | GoAPI | API key, aggregator | [link](https://api.getgoapi.com) | — |
| `gigachat` | `gigachat` | GigaChat (Sber) | API key | [link](https://developers.sber.ru) | — |
| `github-models` | `ghm` | GitHub Models | API key | [link](https://github.com/marketplace/models) | Create a GitHub PAT with 'models: read' scope at github.com/settings/tokens |
| `gitlab` | `gitlab` | GitLab Duo PAT | API key | [link](https://docs.gitlab.com/user/duo_agent_platform/code_suggestions/) | GitLab personal access token for the public Code Suggestions API. Configure a self-hosted base URL when not using gitlab.com. |
| `gitlawb` | `glb` | Gitlawb Opengateway (MiMo) | API key | [link](https://opengateway.gitlawb.com) | Free tier available — no credit card required |
| `gitlawb-gmi` | `glb-gmi` | Gitlawb Opengateway (GMI Cloud) | API key | [link](https://opengateway.gitlawb.com) | Free tier available — no credit card required |
| `glhf` | `glhf` | GLHF Chat | API key, aggregator | [link](https://glhf.chat) | Bearer API key for the GLHF OpenAI-compatible gateway. |
| `glm` | `glm` | GLM Coding | API key | [link](https://z.ai/subscribe) | — |
| `glm-cn` | `glmcn` | GLM Coding (China) | API key | [link](https://open.bigmodel.cn) | — |
| `glmt` | `glmt` | GLM Thinking | API key | [link](https://open.bigmodel.cn) | — |
| `groq` | `groq` | Groq | API key | [link](https://groq.com) | Free tier: 30 RPM / 14.4K RPD — no credit card |
| `hackclub` | `hc` | Hackclub AI | API key, aggregator | [link](https://ai.hackclub.com) | Sign in with your Hack Club account at ai.hackclub.com. |
| `haiper` | `hp` | Haiper | API key, video | [link](https://haiper.ai) | Get API key at haiper.ai/haiper-api |
| `heroku` | `heroku` | Heroku AI | API key, enterprise | [link](https://www.heroku.com) | — |
| `huggingchat` | `huggingchat` | HuggingChat | API key | [link](https://huggingface.co/chat) | No API key required for basic access. |
| `huggingface` | `hf` | HuggingFace | API key | [link](https://huggingface.co) | Free Inference API for thousands of models (Whisper, VITS, SDXL…) |
| `hyperbolic` | `hyp` | Hyperbolic | API key | [link](https://hyperbolic.xyz) | $1-5 trial credits on signup for serverless inference |
| `ideogram` | `ideo` | Ideogram | API key | [link](https://ideogram.ai) | Get API key at ideogram.ai/docs/api |
| `iflytek` | `iflytek` | iFlytek Spark | API key | [link](https://xinghuo.xfyun.cn) | Get API key at console.xfyun.cn |
| `inclusionai` | `inclusion` | InclusionAI | API key | [link](https://inclusionai.com) | Get API key at inclusionai.com |
| `inference-net` | `inet` | Inference.net | API key | [link](https://inference.net) | $25 free credits on signup plus research grants available |
| `jina-ai` | `jina` | Jina AI | API key, embed/rerank | [link](https://jina.ai) | Bearer API key for the Jina AI rerank API. |
| `jina-reader` | `jr` | Jina Reader | API key | [link](https://jina.ai/reader) | — |
| `kie` | `kie` | KIE.AI | API key | [link](https://kie.ai) | — |
| `kilo-gateway` | `kg` | Kilo Gateway | API key, aggregator | [link](https://kilo.ai) | — |
| `kimi` | `kimi` | Kimi | API key | [link](https://platform.moonshot.ai) | — |
| `kimi-coding-apikey` | `kmca` | Kimi Coding (API Key) | API key | [link](https://www.kimi.com/code) | — |
| `kluster` | `kluster` | Kluster AI | API key | [link](https://kluster.ai) | $5 free credits on signup - DeepSeek R1, Llama 4 Maverick/Scout, Qwen3 235B |
| `lambda-ai` | `lambda` | Lambda AI | API key | [link](https://lambda.ai) | — |
| `laozhang` | `lz` | LaoZhang AI | API key, aggregator | [link](https://api.laozhang.ai) | — |
| `leonardo` | `leo` | Leonardo AI | API key, video | [link](https://leonardo.ai) | Get API key at leonardo.ai/developer |
| `liquid` | `liquid` | Liquid AI | API key | [link](https://liquid.ai) | Get API key at liquid.ai |
| `llamagate` | `llamagate` | LlamaGate | API key | [link](https://llamagate.ai) | — |
| `llm7` | `llm7` | LLM7.io | API key | [link](https://llm7.io) | No signup required - 2 req/s, 20 RPM, 100 req/hr free tier |
| `longcat` | `lc` | LongCat AI | API key | [link](https://longcat.chat/platform/docs) | Free: 5M tokens/day on LongCat-2.0-Preview (Flash models retired 2026-05-29); up to 120M/day via feedback. |
| `maritalk` | `maritalk` | Maritalk | API key | [link](https://www.maritaca.ai) | — |
| `meta-llama` | `meta` | Meta Llama API | API key | [link](https://llama.developer.meta.com) | — |
| `minimax` | `minimax` | Minimax Coding | API key, video | [link](https://www.minimax.io) | — |
| `minimax-cn` | `minimax-cn` | Minimax (China) | API key | [link](https://www.minimaxi.com) | — |
| `mistral` | `mistral` | Mistral | API key | [link](https://mistral.ai) | Free Experiment tier: rate-limited access to all models, no credit card required |
| `modal` | `mdl` | Modal | API key, enterprise | [link](https://modal.com/docs) | Use the bearer token that protects your Modal deployment, if enabled. Base URL should point to your OpenAI-compatible Modal app, for example https://<workspace>--<app>.modal.run/v1. |
| `monsterapi` | `monster` | MonsterAPI | API key | [link](https://monsterapi.ai) | Get API key at monsterapi.ai |
| `moonshot` | `moonshot` | Moonshot AI | API key | [link](https://platform.moonshot.ai) | — |
| `morph` | `morph` | Morph | API key | [link](https://morphllm.com) | Free tier: 250K credits/month, $0 |
| `nanogpt` | `nanogpt` | NanoGPT | API key | [link](https://nano-gpt.com) | — |
| `nebius` | `nebius` | Nebius AI | API key | [link](https://nebius.com) | ~$1 trial credits on signup for API testing |
| `nlpcloud` | `nlpc` | NLP Cloud | API key | [link](https://docs.nlpcloud.com) | Use your NLP Cloud API key in Authorization: Token <key>. OmniRoute targets the chatbot endpoint on https://api.nlpcloud.io/v1/gpu/<model>/chatbot by default. |
| `nomic` | `nomic` | Nomic | API key | [link](https://nomic.ai) | Get API key at atlas.nomic.ai |
| `nous-research` | `nous` | Nous Research | API key | [link](https://portal.nousresearch.com/help) | Use your Nous Portal API key. OmniRoute targets the official OpenAI-compatible inference endpoint at https://inference-api.nousresearch.com/v1. |
| `novita` | `novita` | Novita AI | API key, aggregator | [link](https://novita.ai) | $0.50 trial credits on signup (valid about 1 year) |
| `nscale` | `nscale` | nScale | API key | [link](https://nscale.com) | $5 free credits on signup for inference testing |
| `nvidia` | `nvidia` | NVIDIA NIM | API key | [link](https://build.nvidia.com) | Free dev access: ~40 RPM, 70+ models (Kimi K2.5, GLM 4.7, DeepSeek V3.2...) |
| `oci` | `oci` | OCI Generative AI | API key, enterprise | [link](https://www.oracle.com/artificial-intelligence/generative-ai) | Use your OCI Generative AI API key or IAM bearer token. Base URL can be https://inference.generativeai.<region>.oci.oraclecloud.com/openai/v1/. |
| `ollama-cloud` | `ollamacloud` | Ollama Cloud | API key | [link](https://ollama.com/settings/api-keys) | — |
| `openai` | `openai` | OpenAI | API key | [link](https://platform.openai.com) | — |
| `opencode-go` | `opencode-go` | OpenCode Go | API key | [link](https://opencode.ai/go) | — |
| `opencode-zen` | `opencode-zen` | OpenCode Zen | API key | [link](https://opencode.ai/zen) | — |
| `openrouter` | `openrouter` | OpenRouter | API key, aggregator | [link](https://openrouter.ai) | Free models at $0/token with :free suffix - 20 RPM / 200 RPD |
| `ovhcloud` | `ovh` | OVHcloud AI | API key | [link](https://www.ovhcloud.com) | — |
| `perplexity` | `pplx` | Perplexity | API key | [link](https://www.perplexity.ai) | — |
| `phind` | `phind` | Phind | API key | [link](https://phind.com) | Get API key at phind.com |
| `piapi` | `pi` | PiAPI | API key, aggregator | [link](https://piapi.ai) | — |
| `poe` | `poe` | Poe | API key, aggregator | [link](https://creator.poe.com/api-reference) | Bearer API key for the Poe OpenAI-compatible API. |
| `pollinations` | `pol` | Pollinations AI | API key, video | [link](https://pollinations.ai) | No API key required for free public endpoint. Optional Spore tier: ~0.01 pollen/hour. |
| `predibase` | `predibase` | Predibase | API key | [link](https://predibase.com) | $25 free trial credits (30-day validity) |
| `publicai` | `publicai` | PublicAI | API key | [link](https://publicai.co) | Requires an API key — one-time signup credit, then paid |
| `puter` | `pu` | Puter AI | API key | [link](https://puter.com) | Get token at puter.com/dashboard → Copy Auth Token |
| `qianfan` | `qianfan` | Baidu Qianfan | API key | [link](https://cloud.baidu.com/product/wenxinworkshop) | — |
| `recraft` | `recraft` | Recraft | API key, image | [link](https://recraft.ai) | — |
| `reka` | `reka` | Reka | API key | [link](https://docs.reka.ai/chat/overview) | Use your Reka API key. OmniRoute supports the OpenAI-compatible base URL https://api.reka.ai/v1 and sends both Authorization and X-Api-Key headers for compatibility. |
| `runwayml` | `runway` | Runway | API key, video | [link](https://docs.dev.runwayml.com) | Use your Runway API key in Authorization: Bearer <key>. OmniRoute targets the current Runway API at https://api.dev.runwayml.com/v1 and sends the required X-Runway-Version header automatically. |
| `sambanova` | `samba` | SambaNova | API key | [link](https://sambanova.ai) | $5 free credits on signup (30-day validity), no credit card required |
| `sap` | `sap` | SAP Generative AI Hub | API key, enterprise | [link](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/generative-ai-hub-in-sap-ai-core) | Use your SAP AI Core bearer token. Base URL can be your AI_API_URL root or a deploymentUrl from Generative AI Hub. |
| `scaleway` | `scw` | Scaleway AI | API key | [link](https://www.scaleway.com/en/ai/generative-apis) | 1M free tokens for new accounts — EU/GDPR compliant (Paris), Qwen3 235B & Llama 70B |
| `sensenova` | `sensenova` | SenseNova | API key | [link](https://platform.sensenova.cn) | Get API key at platform.sensenova.cn |
| `siliconflow` | `siliconflow` | SiliconFlow | API key | [link](https://cloud.siliconflow.com) | $1 free credits plus permanently free models after identity verification |
| `snowflake` | `snowflake` | Snowflake Cortex | API key, enterprise | [link](https://www.snowflake.com) | — |
| `sparkdesk` | `sparkdesk` | SparkDesk | API key | [link](https://xinghuo.xfyun.cn) | Get API key at console.xfyun.cn |
| `stability-ai` | `stability` | Stability AI | API key, image | [link](https://stability.ai) | — |
| `stepfun` | `stepfun` | StepFun | API key | [link](https://stepfun.com) | Get API key at platform.stepfun.com |
| `suno` | `suno` | Suno | API key | [link](https://suno.ai) | Paste session cookie from suno.ai (Clerk auth) |
| `synthetic` | `synthetic` | Synthetic | API key, aggregator | [link](https://synthetic.new) | — |
| `tencent` | `tencent` | Tencent Hunyuan | API key | [link](https://hunyuan.tencent.com) | Get API key at console.cloud.tencent.com |
| `thebai` | `thebai` | TheB.AI | API key, aggregator | [link](https://theb.ai) | Bearer API key for the TheB.AI OpenAI-compatible gateway. |
| `together` | `together` | Together AI | API key, video | [link](https://www.together.ai) | $25 signup credits + 3 permanently free models: Llama 3.3 70B, Vision, DeepSeek-R1 distill |
| `topaz` | `topaz` | Topaz | API key, image | [link](https://topazlabs.com) | — |
| `udio` | `udio` | Udio | API key | [link](https://udio.com) | Paste session cookie from udio.com (Supabase auth) |
| `uncloseai` | `unc` | UncloseAI | API key | [link](https://uncloseai.com) | No auth required. API accepts any non-empty string as key for identification. |
| `upstage` | `upstage` | Upstage | API key | [link](https://www.upstage.ai) | — |
| `v0-vercel` | `v0` | v0 (Vercel) | API key | [link](https://v0.dev) | — |
| `venice` | `venice` | Venice.ai | API key | [link](https://venice.ai) | — |
| `vercel-ai-gateway` | `vag` | Vercel AI Gateway | API key, aggregator | [link](https://vercel.com/docs/ai-gateway) | — |
| `vertex` | `vertex` | Vertex AI | API key, enterprise | [link](https://cloud.google.com/vertex-ai) | Provide Service Account JSON or OAuth access_token |
| `vertex-partner` | `vp` | Vertex AI Partners | API key, enterprise | [link](https://cloud.google.com/vertex-ai) | Provide the same Service Account JSON used for Vertex AI partner models. |
| `volcengine` | `volcengine` | Volcengine | API key | [link](https://www.volcengine.com) | — |
| `voyage-ai` | `voyage` | Voyage AI | API key, embed/rerank | [link](https://www.voyageai.com) | Bearer API key for Voyage AI embeddings and rerank APIs. |
| `wandb` | `wandb` | Weights & Biases Inference | API key | [link](https://wandb.ai) | — |
| `watsonx` | `watsonx` | IBM watsonx.ai Gateway | API key, enterprise | [link](https://www.ibm.com/products/watsonx-ai) | Use your watsonx bearer token. Base URL can be https://<region>.ml.cloud.ibm.com/ml/gateway/v1/ or a self-managed /ml/gateway/v1 endpoint. |
| `xai` | `xai` | xAI (Grok) | API key | [link](https://x.ai) | — |
| `xiaomi-mimo` | `mimo` | Xiaomi MiMo | API key | [link](https://mimo.mi.com) | — |
| `yi` | `yi` | Yi (01.AI) | API key | [link](https://01.ai) | Get API key at platform.lingyiwanwu.com |
| `zai` | `zai` | Z.AI | API key | [link](https://open.bigmodel.cn) | — |
| `zenmux` | `zm` | ZenMux | API key | [link](https://zenmux.ai) | Use your ZenMux API key in Authorization: Bearer <key>. ZenMux is fully OpenAI-compatible. Base URL: https://zenmux.ai/api/v1. |
## API Key Providers (paid / paid-with-free-credits) (122)
## Local Providers (11)
| ID | Alias | Name | Tags | Website | Notes |
| --------------------- | -------------- | -------------------------- | --------------------- | -------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `agentrouter` | `agentrouter` | AgentRouter | API key, aggregator | [link](https://agentrouter.org) | $200 free credits on signup - multi-model routing gateway |
| `ai21` | `ai21` | AI21 Labs | API key | [link](https://www.ai21.com) | $10 trial credits on signup (valid 3 months), no credit card required |
| `aimlapi` | `aiml` | AI/ML API | API key, aggregator | [link](https://aimlapi.com) | $0.025/day free credits — 200+ models (GPT-4o, Claude, Gemini, Llama) via single endpoint |
| `alibaba` | `ali` | Alibaba | API key | [link](https://dashscope-intl.aliyuncs.com) | — |
| `alibaba-cn` | `ali-cn` | Alibaba (China) | API key | [link](https://dashscope.aliyuncs.com) | — |
| `anthropic` | `anthropic` | Anthropic | API key | [link](https://platform.claude.com) | — |
| `azure-ai` | `azure-ai` | Azure AI Foundry | API key, enterprise | [link](https://learn.microsoft.com/azure/ai-foundry) | Use your Azure AI Foundry key. Base URL can be https://<resource>.services.ai.azure.com/openai/v1/ or https://<resource>.openai.azure.com/openai/v1/. |
| `azure-openai` | `azure` | Azure OpenAI | API key, enterprise | [link](https://azure.microsoft.com/products/ai-services/openai-service) | Use your Azure OpenAI API key. Base URL should be your resource endpoint, for example https://my-resource.openai.azure.com. |
| `bailian-coding-plan` | `bcp` | Alibaba Coding Plan | API key | [link](https://www.alibabacloud.com/help/en/model-studio/coding-plan) | — |
| `baseten` | `baseten` | Baseten | API key | [link](https://baseten.co) | $30 free trial credits for GPU inference |
| `bazaarlink` | `bzl` | BazaarLink | API key | [link](https://bazaarlink.ai) | Free tier with auto:free routing — zero-cost inference, no credit card required |
| `bedrock` | `bedrock` | Amazon Bedrock | API key, enterprise | [link](https://aws.amazon.com/bedrock) | Use your Amazon Bedrock API key in Authorization: Bearer <key>. OmniRoute defaults to the OpenAI-compatible bedrock-mantle endpoint in us-east-1; set a regional base URL if your account uses another region or the bedrock-runtime /openai/v1 path. |
| `black-forest-labs` | `bfl` | Black Forest Labs | API key, image | [link](https://blackforestlabs.ai) | — |
| `blackbox` | `bb` | Blackbox AI | API key | [link](https://blackbox.ai) | Free tier: unlimited basic chat plus Minimax-M2.5, no credit card required |
| `bytez` | `bytez` | Bytez | API key | [link](https://bytez.com) | $1 free credits, refreshes every 4 weeks |
| `cablyai` | `cablyai` | CablyAI | API key, aggregator | [link](https://cablyai.com) | Bearer API key for the CablyAI OpenAI-compatible gateway. |
| `cerebras` | `cerebras` | Cerebras | API key | [link](https://inference.cerebras.ai) | Free: 1M tokens/day, 60K TPM — world's fastest inference |
| `chutes` | `chutes` | Chutes.ai | API key, aggregator | [link](https://chutes.ai) | Bearer API key for the Chutes OpenAI-compatible gateway. |
| `clarifai` | `clarifai` | Clarifai | API key, enterprise | [link](https://docs.clarifai.com) | Use your Clarifai PAT or app-specific API key. OmniRoute targets the OpenAI-compatible endpoint at https://api.clarifai.com/v2/ext/openai/v1 and authenticates with Authorization: Key <token>. |
| `cloudflare-ai` | `cf` | Cloudflare Workers AI | API key | [link](https://developers.cloudflare.com/workers-ai) | Requires API Token AND Account ID (found at dash.cloudflare.com) |
| `codestral` | `codestral` | Codestral | API key | [link](https://mistral.ai) | — |
| `cohere` | `cohere` | Cohere | API key | [link](https://cohere.com) | Free Trial: 1,000 API calls/month for testing, no credit card required |
| `command-code` | `cmd` | Command Code | API key | [link](https://commandcode.ai/) | Use a Command Code API key. Requests are sent to Command Code's /alpha/generate endpoint. |
| `completions` | `cpl` | Completions.me | API key | [link](https://completions.me) | Free unlimited access to Claude, GPT, Gemini — no credit card, no rate limits |
| `crof` | `crof` | CrofAI | API key | [link](https://crof.ai) | — |
| `databricks` | `databricks` | Databricks | API key, enterprise | [link](https://www.databricks.com) | — |
| `datarobot` | `datarobot` | DataRobot | API key, enterprise | [link](https://docs.datarobot.com) | Use your DataRobot API token. Optional Base URL can be the account root (for LLM Gateway) or a deployment URL under /api/v2/deployments/<id>. |
| `deepinfra` | `deepinfra` | DeepInfra | API key | [link](https://deepinfra.com) | Free signup credits for API testing and model exploration |
| `deepseek` | `ds` | DeepSeek | API key | [link](https://platform.deepseek.com) | 5M free tokens on signup - no credit card required |
| `empower` | `empower` | Empower | API key, aggregator | [link](https://docs.empower.dev) | Bearer API key for the Empower OpenAI-compatible endpoint. |
| `enally` | `enly` | Enally AI | API key | [link](https://ai.enally.in) | Free for students and developers — no credit card, OTP verification |
| `fal-ai` | `fal` | Fal.ai | API key, image | [link](https://fal.ai) | — |
| `featherless-ai` | `featherless` | Featherless AI | API key | [link](https://featherless.ai) | — |
| `fenayai` | `fenayai` | FenayAI | API key, aggregator | [link](https://fenayai.com) | Bearer API key for the FenayAI OpenAI-compatible gateway. |
| `fireworks` | `fireworks` | Fireworks AI | API key | [link](https://fireworks.ai) | $1 free starter credits on signup for API testing |
| `freetheai` | `fta` | FreeTheAi | API key | [link](https://freetheai.xyz) | Community-run — free forever, no paid tiers, no credit card |
| `friendliai` | `friendli` | FriendliAI | API key | [link](https://friendli.ai) | — |
| `galadriel` | `galadriel` | Galadriel | API key | [link](https://galadriel.com) | — |
| `gemini` | `gemini` | Gemini (Google AI Studio) | API key | [link](https://aistudio.google.com) | Free forever: 1,500 req/day for Gemini 2.5 Flash — no credit card, get key at aistudio.google.com |
| `getgoapi` | `ggo` | GoAPI | API key, aggregator | [link](https://api.getgoapi.com) | — |
| `gigachat` | `gigachat` | GigaChat (Sber) | API key | [link](https://developers.sber.ru) | — |
| `gitlab` | `gitlab` | GitLab Duo PAT | API key | [link](https://docs.gitlab.com/user/duo_agent_platform/code_suggestions/) | GitLab personal access token for the public Code Suggestions API. Configure a self-hosted base URL when not using gitlab.com. |
| `glhf` | `glhf` | GLHF Chat | API key, aggregator | [link](https://glhf.chat) | Bearer API key for the GLHF OpenAI-compatible gateway. |
| `glm` | `glm` | GLM Coding | API key | [link](https://z.ai/subscribe) | — |
| `glm-cn` | `glmcn` | GLM Coding (China) | API key | [link](https://open.bigmodel.cn) | — |
| `glmt` | `glmt` | GLM Thinking | API key | [link](https://open.bigmodel.cn) | — |
| `groq` | `groq` | Groq | API key | [link](https://groq.com) | Free tier: 30 RPM / 14.4K RPD — no credit card |
| `heroku` | `heroku` | Heroku AI | API key, enterprise | [link](https://www.heroku.com) | — |
| `huggingface` | `hf` | HuggingFace | API key | [link](https://huggingface.co) | Free Inference API for thousands of models (Whisper, VITS, SDXL…) |
| `hyperbolic` | `hyp` | Hyperbolic | API key | [link](https://hyperbolic.xyz) | $1-5 trial credits on signup for serverless inference |
| `inference-net` | `inet` | Inference.net | API key | [link](https://inference.net) | $25 free credits on signup plus research grants available |
| `jina-ai` | `jina` | Jina AI | API key, embed/rerank | [link](https://jina.ai) | Bearer API key for the Jina AI rerank API. |
| `kie` | `kie` | KIE.AI | API key | [link](https://kie.ai) | — |
| `kilo-gateway` | `kg` | Kilo Gateway | API key, aggregator | [link](https://kilo.ai) | — |
| `kimi` | `kimi` | Kimi | API key | [link](https://platform.moonshot.ai) | — |
| `kimi-coding-apikey` | `kmca` | Kimi Coding (API Key) | API key | [link](https://www.kimi.com/code) | — |
| `kluster` | `kluster` | Kluster AI | API key | [link](https://kluster.ai) | $5 free credits on signup - DeepSeek R1, Llama 4 Maverick/Scout, Qwen3 235B |
| `lambda-ai` | `lambda` | Lambda AI | API key | [link](https://lambda.ai) | — |
| `laozhang` | `lz` | LaoZhang AI | API key, aggregator | [link](https://api.laozhang.ai) | — |
| `lepton` | `lepton` | Lepton AI | API key | [link](https://lepton.ai) | Free tier available - fast inference on custom hardware |
| `llamagate` | `llamagate` | LlamaGate | API key | [link](https://llamagate.ai) | — |
| `llm7` | `llm7` | LLM7.io | API key | [link](https://llm7.io) | No signup required - 2 req/s, 20 RPM, 100 req/hr free tier |
| `longcat` | `lc` | LongCat AI | API key | [link](https://longcat.chat/platform/docs) | 50M tokens/day (Flash-Lite) + 500K/day (Chat/Thinking) — 100% free while public beta |
| `maritalk` | `maritalk` | Maritalk | API key | [link](https://www.maritaca.ai) | — |
| `meta-llama` | `meta` | Meta Llama API | API key | [link](https://llama.developer.meta.com) | — |
| `minimax` | `minimax` | Minimax Coding | API key | [link](https://www.minimax.io) | — |
| `minimax-cn` | `minimax-cn` | Minimax (China) | API key | [link](https://www.minimaxi.com) | — |
| `mistral` | `mistral` | Mistral | API key | [link](https://mistral.ai) | Free Experiment tier: rate-limited access to all models, no credit card required |
| `modal` | `mdl` | Modal | API key, enterprise | [link](https://modal.com/docs) | Use the bearer token that protects your Modal deployment, if enabled. Base URL should point to your OpenAI-compatible Modal app, for example https://<workspace>--<app>.modal.run/v1. |
| `moonshot` | `moonshot` | Moonshot AI | API key | [link](https://platform.moonshot.ai) | — |
| `morph` | `morph` | Morph | API key | [link](https://morphllm.com) | Free tier: 250K credits/month, $0 |
| `nanobanana` | `nb` | NanoBanana | API key, image | [link](https://nanobananaapi.ai) | — |
| `nanogpt` | `nanogpt` | NanoGPT | API key | [link](https://nano-gpt.com) | — |
| `nebius` | `nebius` | Nebius AI | API key | [link](https://nebius.com) | ~$1 trial credits on signup for API testing |
| `nlpcloud` | `nlpc` | NLP Cloud | API key | [link](https://docs.nlpcloud.com) | Use your NLP Cloud API key in Authorization: Token <key>. OmniRoute targets the chatbot endpoint on https://api.nlpcloud.io/v1/gpu/<model>/chatbot by default. |
| `nous-research` | `nous` | Nous Research | API key | [link](https://portal.nousresearch.com/help) | Use your Nous Portal API key. OmniRoute targets the official OpenAI-compatible inference endpoint at https://inference-api.nousresearch.com/v1. |
| `novita` | `novita` | Novita AI | API key, aggregator | [link](https://novita.ai) | $0.50 trial credits on signup (valid about 1 year) |
| `nscale` | `nscale` | nScale | API key | [link](https://nscale.com) | $5 free credits on signup for inference testing |
| `nvidia` | `nvidia` | NVIDIA NIM | API key | [link](https://build.nvidia.com) | Free dev access: ~40 RPM, 70+ models (Kimi K2.5, GLM 4.7, DeepSeek V3.2...) |
| `oci` | `oci` | OCI Generative AI | API key, enterprise | [link](https://www.oracle.com/artificial-intelligence/generative-ai) | Use your OCI Generative AI API key or IAM bearer token. Base URL can be https://inference.generativeai.<region>.oci.oraclecloud.com/openai/v1/. |
| `ollama-cloud` | `ollamacloud` | Ollama Cloud | API key | [link](https://ollama.com/settings/api-keys) | — |
| `openai` | `openai` | OpenAI | API key | [link](https://platform.openai.com) | — |
| `opencode-go` | `opencode-go` | OpenCode Go | API key | [link](https://opencode.ai/go) | — |
| `opencode-zen` | `opencode-zen` | OpenCode Zen | API key | [link](https://opencode.ai/zen) | — |
| `openrouter` | `openrouter` | OpenRouter | API key, aggregator | [link](https://openrouter.ai) | Free models at $0/token with :free suffix - 20 RPM / 200 RPD |
| `ovhcloud` | `ovh` | OVHcloud AI | API key | [link](https://www.ovhcloud.com) | — |
| `perplexity` | `pplx` | Perplexity | API key | [link](https://www.perplexity.ai) | — |
| `petals` | `petals` | Petals | API key | [link](https://chat.petals.dev) | No API key is required for the public research endpoint. Leave the field blank, or provide a bearer token if your self-hosted Petals gateway uses auth. |
| `piapi` | `pi` | PiAPI | API key, aggregator | [link](https://piapi.ai) | — |
| `poe` | `poe` | Poe | API key, aggregator | [link](https://creator.poe.com/api-reference) | Bearer API key for the Poe OpenAI-compatible API. |
| `pollinations` | `pol` | Pollinations AI | API key | [link](https://pollinations.ai) | No API key required for free public endpoint. Optional Spore tier: ~0.01 pollen/hour. |
| `predibase` | `predibase` | Predibase | API key | [link](https://predibase.com) | $25 free trial credits (30-day validity) |
| `publicai` | `publicai` | PublicAI | API key | [link](https://publicai.co) | Free community inference tier for testing |
| `puter` | `pu` | Puter AI | API key | [link](https://puter.com) | Get token at puter.com/dashboard → Copy Auth Token |
| `qianfan` | `qianfan` | Baidu Qianfan | API key | [link](https://cloud.baidu.com/product/wenxinworkshop) | — |
| `recraft` | `recraft` | Recraft | API key, image | [link](https://recraft.ai) | — |
| `reka` | `reka` | Reka | API key | [link](https://docs.reka.ai/chat/overview) | Use your Reka API key. OmniRoute supports the OpenAI-compatible base URL https://api.reka.ai/v1 and sends both Authorization and X-Api-Key headers for compatibility. |
| `runwayml` | `runway` | Runway | API key, video | [link](https://docs.dev.runwayml.com) | Use your Runway API key in Authorization: Bearer <key>. OmniRoute targets the current Runway API at https://api.dev.runwayml.com/v1 and sends the required X-Runway-Version header automatically. |
| `sambanova` | `samba` | SambaNova | API key | [link](https://sambanova.ai) | $5 free credits on signup (30-day validity), no credit card required |
| `sap` | `sap` | SAP Generative AI Hub | API key, enterprise | [link](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/generative-ai-hub-in-sap-ai-core) | Use your SAP AI Core bearer token. Base URL can be your AI_API_URL root or a deploymentUrl from Generative AI Hub. |
| `scaleway` | `scw` | Scaleway AI | API key | [link](https://www.scaleway.com/en/ai/generative-apis) | 1M free tokens for new accounts — EU/GDPR compliant (Paris), Qwen3 235B & Llama 70B |
| `siliconflow` | `siliconflow` | SiliconFlow | API key | [link](https://cloud.siliconflow.com) | $1 free credits plus permanently free models after identity verification |
| `snowflake` | `snowflake` | Snowflake Cortex | API key, enterprise | [link](https://www.snowflake.com) | — |
| `stability-ai` | `stability` | Stability AI | API key, image | [link](https://stability.ai) | — |
| `synthetic` | `synthetic` | Synthetic | API key, aggregator | [link](https://synthetic.new) | — |
| `thebai` | `thebai` | TheB.AI | API key, aggregator | [link](https://theb.ai) | Bearer API key for the TheB.AI OpenAI-compatible gateway. |
| `together` | `together` | Together AI | API key | [link](https://www.together.ai) | $25 signup credits + 3 permanently free models: Llama 3.3 70B, Vision, DeepSeek-R1 distill |
| `topaz` | `topaz` | Topaz | API key, image | [link](https://topazlabs.com) | — |
| `uncloseai` | `unc` | UncloseAI | API key | [link](https://uncloseai.com) | No auth required. API accepts any non-empty string as key for identification. |
| `upstage` | `upstage` | Upstage | API key | [link](https://www.upstage.ai) | — |
| `v0-vercel` | `v0` | v0 (Vercel) | API key | [link](https://v0.dev) | — |
| `venice` | `venice` | Venice.ai | API key | [link](https://venice.ai) | — |
| `vercel-ai-gateway` | `vag` | Vercel AI Gateway | API key, aggregator | [link](https://vercel.com/docs/ai-gateway) | — |
| `vertex` | `vertex` | Vertex AI | API key, enterprise | [link](https://cloud.google.com/vertex-ai) | Provide Service Account JSON or OAuth access_token |
| `vertex-partner` | `vp` | Vertex AI Partners | API key, enterprise | [link](https://cloud.google.com/vertex-ai) | Provide the same Service Account JSON used for Vertex AI partner models. |
| `volcengine` | `volcengine` | Volcengine | API key | [link](https://www.volcengine.com) | — |
| `voyage-ai` | `voyage` | Voyage AI | API key, embed/rerank | [link](https://www.voyageai.com) | Bearer API key for Voyage AI embeddings and rerank APIs. |
| `wandb` | `wandb` | Weights & Biases Inference | API key | [link](https://wandb.ai) | — |
| `watsonx` | `watsonx` | IBM watsonx.ai Gateway | API key, enterprise | [link](https://www.ibm.com/products/watsonx-ai) | Use your watsonx bearer token. Base URL can be https://<region>.ml.cloud.ibm.com/ml/gateway/v1/ or a self-managed /ml/gateway/v1 endpoint. |
| `xai` | `xai` | xAI (Grok) | API key | [link](https://x.ai) | — |
| `xiaomi-mimo` | `mimo` | Xiaomi MiMo | API key | [link](https://mimo.mi.com) | — |
| `zai` | `zai` | Z.AI | API key | [link](https://open.bigmodel.cn) | — |
## Local Providers (10)
| ID | Alias | Name | Tags | Website | Notes |
| --------------------- | ------------ | ------------------- | ------------------ | --------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| `comfyui` | `comfyui` | ComfyUI | Local | [link](https://github.com/comfyanonymous/ComfyUI) | No API key required. Configure the local ComfyUI base URL (default: http://localhost:8188). |
| `docker-model-runner` | `dmr` | Docker Model Runner | Local, self-hosted | [link](https://docs.docker.com/ai/model-runner/) | API key optional. Configure the local Docker Model Runner OpenAI-compatible base URL (default: http://localhost:12434/v1). |
| `lemonade` | `lemonade` | Lemonade Server | Local, self-hosted | [link](https://lemonade-server.ai) | API key optional. Configure the local Lemonade OpenAI-compatible base URL (default: http://localhost:13305/api/v1). |
| `llamafile` | `llamafile` | Llamafile | Local, self-hosted | [link](https://github.com/Mozilla-Ocho/llamafile) | API key optional. Configure the local Llamafile OpenAI-compatible base URL (default: http://127.0.0.1:8080/v1). |
| `lm-studio` | `lmstudio` | LM Studio | Local, self-hosted | [link](https://lmstudio.ai) | API key optional. Configure the local LM Studio OpenAI-compatible base URL (default: http://localhost:1234/v1). |
| `oobabooga` | `ooba` | oobabooga | Local, self-hosted | [link](https://github.com/oobabooga/text-generation-webui) | API key optional. Configure the local oobabooga OpenAI-compatible base URL (default: http://localhost:5000/v1). |
| `sdwebui` | `sdwebui` | SD WebUI | Local | [link](https://github.com/AUTOMATIC1111/stable-diffusion-webui) | No API key required. Configure the local WebUI base URL (default: http://localhost:7860). |
| `triton` | `triton` | NVIDIA Triton | Local, self-hosted | [link](https://developer.nvidia.com/triton-inference-server) | API key optional. Configure the Triton OpenAI-compatible base URL (default: http://localhost:8000/v1). |
| `vllm` | `vllm` | vLLM | Local, self-hosted | [link](https://github.com/vllm-project/vllm) | API key optional. Configure the local vLLM OpenAI-compatible base URL (default: http://localhost:8000/v1). |
| `xinference` | `xinference` | XInference | Local, self-hosted | [link](https://inference.readthedocs.io) | API key optional. Configure the local XInference OpenAI-compatible base URL (default: http://localhost:9997/v1). |
| ID | Alias | Name | Tags | Website | Notes |
| --------------------- | ------------ | ------------------- | ------------------ | --------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `comfyui` | `comfyui` | ComfyUI | Local | [link](https://github.com/comfyanonymous/ComfyUI) | No API key required. Configure the local ComfyUI base URL (default: http://localhost:8188). |
| `docker-model-runner` | `dmr` | Docker Model Runner | Local, self-hosted | [link](https://docs.docker.com/ai/model-runner/) | API key optional. Configure the local Docker Model Runner OpenAI-compatible base URL (default: http://localhost:12434/v1). |
| `lemonade` | `lemonade` | Lemonade Server | Local, self-hosted | [link](https://lemonade-server.ai) | API key optional. Configure the local Lemonade OpenAI-compatible base URL (default: http://localhost:13305/api/v1). |
| `llama-cpp` | `llamacpp` | llama.cpp | Local, self-hosted | [link](https://github.com/ggml-org/llama.cpp) | API key optional (use any value, e.g. sk-no-key-required). Configure the llama-server OpenAI-compatible base URL (default: http://127.0.0.1:8080/v1). Note: if Llamafile is also installed, both default to port 8080 — run only one at a time or override the port. |
| `llamafile` | `llamafile` | Llamafile | Local, self-hosted | [link](https://github.com/Mozilla-Ocho/llamafile) | API key optional. Configure the local Llamafile OpenAI-compatible base URL (default: http://127.0.0.1:8080/v1). |
| `lm-studio` | `lmstudio` | LM Studio | Local, self-hosted | [link](https://lmstudio.ai) | API key optional. Configure the local LM Studio OpenAI-compatible base URL (default: http://localhost:1234/v1). |
| `oobabooga` | `ooba` | oobabooga | Local, self-hosted | [link](https://github.com/oobabooga/text-generation-webui) | API key optional. Configure the local oobabooga OpenAI-compatible base URL (default: http://localhost:5000/v1). |
| `sdwebui` | `sdwebui` | SD WebUI | Local | [link](https://github.com/AUTOMATIC1111/stable-diffusion-webui) | No API key required. Configure the local WebUI base URL (default: http://localhost:7860). |
| `triton` | `triton` | NVIDIA Triton | Local, self-hosted | [link](https://developer.nvidia.com/triton-inference-server) | API key optional. Configure the Triton OpenAI-compatible base URL (default: http://localhost:8000/v1). |
| `vllm` | `vllm` | vLLM | Local, self-hosted | [link](https://github.com/vllm-project/vllm) | API key optional. Configure the local vLLM OpenAI-compatible base URL (default: http://localhost:8000/v1). |
| `xinference` | `xinference` | XInference | Local, self-hosted | [link](https://inference.readthedocs.io) | API key optional. Configure the local XInference OpenAI-compatible base URL (default: http://localhost:9997/v1). |
## Search Providers (11)
@@ -236,10 +280,11 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
| `inworld` | `inworld` | Inworld | Audio | [link](https://inworld.ai) | — |
| `playht` | `playht` | PlayHT | Audio | [link](https://play.ht) | — |
## Upstream Proxy Providers (1)
## Upstream Proxy Providers (2)
| ID | Alias | Name | Tags | Website | Notes |
| ------------- | ----- | ----------- | -------------- | ---------------------------------------------------- | ----- |
| `9router` | `nr` | 9router | Upstream proxy | [link](https://www.npmjs.com/package/9router) | — |
| `cliproxyapi` | `cpa` | CLIProxyAPI | Upstream proxy | [link](https://github.com/router-for-me/CLIProxyAPI) | — |
## Cloud Agent Providers (3)

2
Providers-Guide.md

@@ -220,4 +220,4 @@ Go to Providers → click on the provider → click **Disconnect**.
- **[Auto-Combo Guide](./AUTO-COMBO-GUIDE.md)** — Let OmniRoute pick the best AI for you
- **[Free Tiers Guide](./FREE-TIERS-GUIDE.md)** — Get free AI with no credit card
- **[Troubleshooting](./TROUBLESHOOTING.md)** — Fix common issues
- **[Provider Reference](../reference/PROVIDER_REFERENCE.md)** — Full list of 177 providers
- **[Provider Reference](../reference/PROVIDER_REFERENCE.md)** — Full list of 226 providers

1
Providers.md

@@ -1,5 +1,6 @@
> 🌍 [View in other languages](Languages)
# Providers — Claude Web
## claude-web

217
Proxy-Guide.md

@@ -596,6 +596,223 @@ CREATE TABLE proxy_assignments (
---
## Proxy Health Checking (v3.8.16+)
OmniRoute's **proxy fast-fail** mechanism (`src/lib/proxyHealth.ts`) detects dead proxies in <2s via a quick TCP connection check, then **caches the result** to avoid per-request overhead.
### How It Works
```
Request ──▶ ProxyHealthCache.get(url)
│
├─ Cache hit + fresh? ──▶ return cached status
│
└─ Cache miss / stale? ──▶ TCP connect to host:port
(timeout: FAST_FAIL_TIMEOUT_MS)
──▶ cache for HEALTH_CACHE_TTL_MS
──▶ return result
```
Without this, a dead proxy would block every request for the full `PROXY_TIMEOUT_MS` (default 30s) before failing.
### Tunable Environment Variables
| Variable | Default | Purpose |
|----------|---------|---------|
| `PROXY_FAST_FAIL_TIMEOUT_MS` | `2000` | TCP connection timeout per health check |
| `PROXY_HEALTH_CACHE_TTL_MS` | `30000` | How long a health result is cached |
**Recommended values:**
| Scenario | Fast-fail timeout | Cache TTL | Reasoning |
|----------|-------------------|-----------|-----------|
| High-throughput API gateway | 1500ms | 60000ms | Aggressive fail-fast, longer cache to reduce checks |
| Geo-distributed nodes | 3000ms | 15000ms | Slower networks need more time; shorter cache for fast failover |
| Dev / testing | 1000ms | 10000ms | Quick iteration on local proxies |
| Stealth / anti-detection | 2500ms | 45000ms | Avoid rapid probing that could trigger rate limits |
### Inspecting Proxy Health
```ts
import { getAllProxyHealthStatuses, invalidateProxyHealth } from "omniroute/proxyHealth";
const statuses = getAllProxyHealthStatuses();
for (const s of statuses) {
console.log(`${s.proxyUrl} → healthy=${s.healthy}, stale=${s.stale}`);
}
// Force re-check a specific proxy
invalidateProxyHealth("http://user:pass@1.2.3.4:8080");
```
The `stale` flag is `true` when the cache entry has exceeded `HEALTH_CACHE_TTL_MS` and the next request will trigger a fresh check.
### Per-Proxy Type Defaults
The health check uses sensible defaults based on the URL scheme:
| Scheme | Default port |
|--------|-------------|
| `http://` | 8080 |
| `https://` | 443 |
| `socks5://` / `socks5h://` | 1080 |
Custom ports in the URL (`http://host:9999`) always take precedence over the scheme default.
---
## Proxy Analytics & Observability
OmniRoute tracks per-proxy usage to help operators diagnose routing patterns, latency spikes, and recurring failures.
### What's Tracked
For every request through a configured proxy, OmniRoute records:
| Metric | Description |
|--------|-------------|
| `proxy_url` | Full proxy URL (with auth credentials masked) |
| `provider` | Upstream provider ID (openai, anthropic, etc.) |
| `latency_ms` | Total round-trip time including proxy handshake |
| `connect_ms` | TCP connect time only |
| `status` | HTTP status code from upstream |
| `error` | Error class if request failed |
| `timestamp` | ISO 8601 UTC |
### Accessing the Data
```bash
# Recent proxy events
curl -H "Authorization: Bearer $OMNIROUTE_KEY" \
"http://localhost:20128/api/usage/proxy-logs?limit=100"
```
The real endpoint is `/api/usage/proxy-logs` (see `src/app/api/usage/proxy-logs/route.ts`). This endpoint supports:
- `GET /api/usage/proxy-logs` — retrieve proxy logs
- `DELETE /api/usage/proxy-logs` — clear all proxy logs
Aggregate stats can be queried directly from the `proxy_logs` table via SQL if needed. The dashboard UI may offer aggregate views.
### Common Patterns
**Detect a flapping proxy** (alternates between success/failure):
```sql
SELECT proxy_url,
COUNT(*) AS total,
SUM(CASE WHEN status >= 500 THEN 1 ELSE 0 END) AS errors,
ROUND(100.0 * SUM(CASE WHEN status >= 500 THEN 1 ELSE 0 END) / COUNT(*), 1) AS error_pct
FROM proxy_logs
WHERE timestamp > datetime('now', '-1 hour')
GROUP BY proxy_url
HAVING error_pct > 5
ORDER BY error_pct DESC;
```
**Find slow proxies** (p95 latency > 2s):
```sql
WITH ranked AS (
SELECT proxy_url, latency_ms,
PERCENT_RANK() OVER (PARTITION BY proxy_url ORDER BY latency_ms) AS pct
FROM proxy_logs
WHERE timestamp > datetime('now', '-24 hour')
)
SELECT proxy_url, latency_ms
FROM ranked
WHERE pct >= 0.95
ORDER BY latency_ms DESC;
```
---
## Rotation Strategy Decision Tree
When multiple proxies are assigned to a scope, OmniRoute uses a **rotation strategy** to pick which one to use for each request. The strategy is configured at the scope level (global, per-provider, per-account, per-combo).
### Available Strategies
| Strategy | When to use | Trade-off |
|----------|-------------|-----------|
| `quality` (default) | Production with mixed-quality proxies | Favors high-rated proxies; may starve low-rated ones |
| `random` | Load distribution, privacy | Even distribution; ignores quality signals |
| `sequential` | Debugging, deterministic testing | Cycles through proxies in order; easy to reason about |
### Decision Tree
```
Do you have quality scores for your proxies?
│
┌───────────┴───────────┐
│ │
YES NO
│ │
Are all proxies │
roughly equal │
in quality? │
│ │
┌────┴────┐ │
│ │ │
YES NO Use
│ │ `random`
│ │ (even spread
│ │ builds quality
│ │ data over time)
│ │
│ Use `quality`
│ (best for
│ mixed quality)
│
Use `random`
(spread load
evenly)
```
### Configuring Rotation Strategy
```ts
import { rotateOneproxyProxy } from "omniroute/oneproxyRotator";
// In a one-off script
const proxy = await rotateOneproxyProxy({ strategy: "quality" });
if (proxy) {
console.log(`Selected: ${proxy.host}:${proxy.port}, quality=${proxy.qualityScore}`);
}
```
### Resetting Sequential Index
When using `sequential` strategy, the internal index accumulates. To reset:
```ts
import { resetSequentialIndex } from "omniroute/oneproxyRotator";
resetSequentialIndex();
```
Useful when:
- Restarting a load test
- Recovering from a proxy outage (so you don't cycle through dead ones first)
- Manually rebalancing after adding new proxies
### Marking a Proxy as Failed
When a proxy consistently fails, mark it manually so the rotator will skip it:
```ts
import { failOneproxyProxy } from "omniroute/oneproxyRotator";
const removed = await failOneproxyProxy("1.2.3.4", 8080);
if (removed) {
console.log("Proxy marked as failed; rotator will skip it");
}
```
The proxy is **not deleted** — it's marked unhealthy and won't be selected until the next successful health check (via `proxyHealth.ts`) or manual reset.
---
> 📖 **Related documentation:**
>
> - [User Guide](../guides/USER_GUIDE.md) — General setup and configuration

2
Quality-Gates.md

@@ -102,7 +102,7 @@ Runs after `build`. Blocks merge on failure.
| Suite | Validates | Blocking |
| ---------------- | ------------------------------------------------------- | -------------------------------------------------------------------------- |
| `test:vitest` | MCP server (43 tools), autoCombo, cache — vitest runner | Yes |
| `test:vitest` | MCP server (87 tools), autoCombo, cache — vitest runner | Yes |
| `test:vitest:ui` | UI component tests — vitest runner | **Advisory** (`continue-on-error: true`) — failing until Fase 6A UI triage |
---

364
RTK-Compression.md

@@ -235,3 +235,367 @@ node --import tsx/esm --test \
4. Add command detection coverage when introducing a new output class.
5. Run the verify and broad RTK gates.
6. If the filter is project-local, commit `.rtk/filters.json` and refresh `.rtk/trust.json` only after review.
---
## Intensity Levels (v3.8.16+)
RTK supports **3 intensity levels** that trade off between **compression aggressiveness** and **safety**. The level is set via `config.intensity` in the engine config.
### The 3 Levels
| Level | Truncation threshold | Token savings | Risk | Best for |
|-------|---------------------|---------------|------|----------|
| `minimal` | 24 lines per section | ~20-40% | Very low | Production with critical context |
| `standard` (default) | 24 lines per section | ~50-70% | Low | Daily coding sessions |
| `aggressive` | 16 lines per section | ~70-90% | Medium | Long sessions, max savings |
### Where the Truncation Happens
The truncation threshold affects `lineFilter.ts`:
```ts
// From open-sse/services/compression/engines/rtk/index.ts:329-330
config.intensity === "aggressive" ? 16 : 24,
config.intensity === "aggressive" ? 16 : 24,
```
Both the **head** and **tail** of each section are preserved; middle content is dropped when truncation kicks in.
### What Stays vs. What Gets Cut
| Content | minimal | standard | aggressive |
|---------|---------|----------|------------|
| Errors / stack traces | ✅ preserved | ✅ preserved | ✅ preserved |
| Test failures | ✅ preserved | ✅ preserved | ✅ preserved |
| Build errors | ✅ preserved | ✅ preserved | ✅ preserved |
| Test passes (verbose) | ✅ preserved | 🟡 collapsed | 🟡 collapsed |
| Routine output (info logs) | 🟡 collapsed | 🟡 collapsed | ❌ dropped |
| Progress bars | 🟡 collapsed | ❌ dropped | ❌ dropped |
| Banner / ASCII art | 🟡 collapsed | ❌ dropped | ❌ dropped |
### Choosing the Right Intensity
```
Is losing context catastrophic?
│
┌───────────┼───────────┐
│ │ │
YES NO NOT SURE
│ │ │
▼ │ │
minimal │ │
│ │ │
│ ▼ ▼
│ How critical Try `standard` first
│ is throughput? (works for 80% of
│ │ cases)
│ ┌────┴────┐
│ │ │
│ LOW HIGH
│ │ │
│ ▼ ▼
│ standard aggressive
│ │ │
└──────┴─────────┘
```
### Configuring Intensity
**Per-combo** (in combo config):
```json
{
"combo": "my-coding-combo",
"routing": { /* ... */ },
"compression": {
"engine": "rtk",
"intensity": "aggressive"
}
}
```
**Programmatically**:
`rtkEngine` (`@omniroute/open-sse/services/compression/engines/rtk`) is a
`CompressionEngine` and has no `updateConfig` method. Update an engine's config
through the registry helper instead:
```ts
import { updateEngineConfig } from "@omniroute/open-sse/services/compression/engines/registry";
updateEngineConfig("rtk", { intensity: "aggressive" });
```
### Verifying the Effect
Use the **Verify Gate** (see below) to confirm your filter is safe at your chosen intensity:
```ts
import { runRtkFilterTests } from "omniroute/compression/engines/rtk/verify";
const result = runRtkFilterTests({ intensity: "aggressive" });
if (!result.passed) {
console.error("Filters failed at aggressive intensity");
}
```
---
## Custom Filter Development (v3.8.16+)
The `engines/rtk/filters/` directory contains **49+ built-in filter JSON files**. You can add your own to compress output from custom tools not covered by the defaults.
### Filter Schema (Zod)
```ts
{
"id": "string", // Required. Filter identifier (kebab-case, e.g., "python-traceback")
"label": "string", // Required. Human-readable filter name
"description": "string", // Optional (default: ""). Short description of what filter does
"category": "git|test|build|shell|docker|package|infra|cloud|generic",
"priority": number, // Optional (0-100, default: 50). Execution order (higher = first)
"match": {
"commands": ["string"], // Command names to match (e.g., "python", "pytest")
"patterns": ["string"], // Regex patterns to match output
"outputTypes": ["string"] // Detected output classes (e.g., "test-failure")
},
"rules": {
"stripAnsi": boolean, // Optional (default: false). Strip ANSI color codes
"replace": [ // Find-and-replace rules (default: [])
{ "pattern": "regex", "replacement": "..." }
],
"matchOutput": [ // Short-circuit on pattern match (default: [])
{
"pattern": "regex",
"message": "short summary",
"unless": "regex" // Skip if this pattern matches
}
],
"includePatterns": ["string"], // Lines to keep (regex patterns, default: [])
"dropPatterns": ["string"], // Lines to drop (regex patterns, default: [])
"collapsePatterns": ["string"], // Lines to collapse to single occurrence (default: [])
"deduplicate": boolean, // Optional (default: false). Remove duplicate lines
"truncateLineAt": number, // Optional (default: 0). Truncate lines to max chars
"maxLines": number, // Optional (default: 0). Hard cap on total lines
"headLines": number, // Optional (default: 20). Keep first N lines of matched output
"tailLines": number, // Optional (default: 20). Keep last N lines of matched output
"onEmpty": "string", // Optional (default: ""). Fallback message if all lines filtered
"filterStderr": boolean // Optional (default: false). Also filter stderr output
},
"preserve": {
"errorPatterns": ["string"], // Patterns that must always be preserved (default: [])
"summaryPatterns": ["string"] // Patterns for final summary line (default: [])
},
"tests": [ // Inline tests for verification (default: [])
{
"name": "string", // Required. Test name
"input": "sample output", // Required. Sample input text
"expected": "expected output", // Required. Expected compressed output
"command": "optional command" // Optional. Command context
}
]
}
```
### Example: Python Traceback Filter
```json
{
"id": "python-traceback",
"label": "Python Traceback Filter",
"description": "Compresses Python tracebacks to essential file/line locations and error type",
"category": "test",
"priority": 60,
"match": {
"commands": ["python", "python3", "pytest", "uv", "poetry"],
"patterns": ["Traceback \\(most recent call last\\)", "Error", "Exception"],
"outputTypes": ["error-traceback"]
},
"rules": {
"stripAnsi": true,
"includePatterns": [
"Traceback \\(most recent call last\\)",
"^\\s*File \".+\", line \\d+",
"^\\s*[A-Z][a-zA-Z]+Error:",
"^\\s*[A-Z][a-zA-Z]+Exception"
],
"dropPatterns": [
"site-packages/",
"^\\s+[a-z_]+\\([^)]*\\)$"
],
"headLines": 5,
"tailLines": 3,
"maxLines": 25,
"filterStderr": true
},
"preserve": {
"errorPatterns": [
"Error:",
"Exception:",
"Traceback"
],
"summaryPatterns": [
"^[A-Z][a-zA-Z]+(?:Error|Exception):"
]
},
"tests": [
{
"name": "preserves-error-type-and-location",
"input": "Traceback (most recent call last):\n File \"app.py\", line 42, in main\n do_thing()\n File \"lib/utils.py\", line 17, in helper\n return 1 / 0\nZeroDivisionError: division by zero",
"expected": "Traceback (most recent call last):\n File \"app.py\", line 42, in main\n File \"lib/utils.py\", line 17, in helper\nZeroDivisionError: division by zero",
"command": "python app.py"
}
]
}
```
### Loading Custom Filters
Place the file in a recognized location:
```
~/.omniroute/rtk/filters/my-filter.json # User-level
<project>/.rtk/filters/my-filter.json # Project-level
```
Filters are loaded automatically on startup via `loadRtkFilters()` in `open-sse/services/compression/engines/rtk/filterLoader.ts`. The loader discovers filters from:
- Built-in catalog: `open-sse/services/compression/engines/rtk/filters/`
- User directory: `~/.omniroute/rtk/filters/`
- Project directory: `<project>/.rtk/filters/`
To load filters programmatically:
```ts
import { loadRtkFilters } from "@omniroute/open-sse/services/compression/engines/rtk/filterLoader";
// Options: customFiltersEnabled (load user/project filters, default on),
// trustProjectFilters, refresh.
const filters = loadRtkFilters({ customFiltersEnabled: true });
```
### Validation
Filters are validated against the Zod schema on load. A filter with bad structure will fail to load and log an error:
```
RTK_FILTER_LOADER: filter "my-filter" failed validation:
- rules.replace.0.pattern: Invalid regex
- match.commands: must not be empty
```
To validate all installed filters, call `runRtkFilterTests()` which is exported from `open-sse/services/compression/engines/rtk/verify.ts`.
### Best Practices
1. **Always include `tests[]`** — they prove your filter works and prevent regressions
2. **Use `matchOutput` for short-circuits** — if a single line tells the story, replace the whole block
3. **Prefer `keep` over `strip`** — explicit "always preserve" rules are safer than "always remove"
4. **Test at all 3 intensity levels** — `minimal` should be a no-op, `aggressive` should still preserve errors
5. **Use the `unless` field** — guard short-circuits with "don't trigger if X is present"
---
## Raw Output Recovery & Verify Gate
When RTK compresses output aggressively, you can **recover the original text** for debugging, audit, or replay.
### How Raw Output Recovery Works
```
Original output (10K tokens)
│
▼
RTK compress (with rawOutput.enabled=true)
│
├─▶ Compressed output (2K tokens) ──▶ to LLM
│
└─▶ Original output (10K tokens) ──▶ stored in DB
(linked by request_id)
```
### Enabling Raw Output Storage
**Per-request** (in combo config):
```json
{
"compression": {
"engine": "rtk",
"intensity": "aggressive",
"rawOutput": {
"enabled": true,
"maxBytes": 1048576 // 1MB cap
}
}
}
```
**Default**: `rawOutput.enabled: false` (saves storage).
### Storage Cost
| Per-request | 1MB cap | 10MB cap |
|-------------|---------|----------|
| Average compressed output | ~5KB | ~5KB |
| Raw output stored | ~50-500KB | ~500KB-5MB |
| With 1000 requests/day | 50-500MB/day | 500MB-5GB/day |
> **Recommendation**: Only enable raw output for **debugging sessions** or **sampled auditing**, not always-on.
### Recovering the Original
```ts
import { readRtkRawOutput } from "omniroute/compression/engines/rtk/rawOutput";
const raw = readRtkRawOutput(pointerId); // pointerId from compression stats
if (raw) {
console.log("Original output:", raw);
}
```
The `pointerId` is returned in `CompressionStats.rtkRawOutputPointers[]` after compression.
See `open-sse/services/compression/engines/rtk/rawOutput.ts:102` for the function signature.
### The Verify Gate
The **RTK Filter Verification** (`open-sse/services/compression/engines/rtk/verify.ts`) validates all filters against their `tests[]` and ensures behavior is correct at all 3 intensity levels.
**Call `runRtkFilterTests()`** to run verification:
```ts
import { runRtkFilterTests } from "open-sse/services/compression/engines/rtk/verify";
const result = runRtkFilterTests();
console.log(`Passed: ${result.outcomes.filter(o => o.passed).length}`);
console.log(`Failed: ${result.outcomes.filter(o => !o.passed).length}`);
if (!result.passed) {
console.error("Filters failed verification");
result.outcomes.filter(o => !o.passed).forEach(o => {
console.error(` - ${o.filterId} / ${o.testName}: expected "${o.expected}", got "${o.actual}"`);
});
}
```
**What it validates**:
1. Every filter loads and passes schema validation
2. Every `tests[]` entry produces expected output
3. `minimal` intensity is a no-op (preserves original, only applies structural filters)
4. `aggressive` intensity preserves errors, test failures, and stack traces
5. Compressed output is never larger than original input
- Source: `open-sse/services/compression/engines/rtk/` (63 files, ~70KB)
- **Before merging a filter change** — always ensure tests pass
- **After upgrading RTK engine** — schema may have changed
- **Periodically in monitoring** — protects against drift in test fixtures
- **When adding a new tool/command family** — proves the new filter works
---
## See Also
- [COMPRESSION_GUIDE.md](./COMPRESSION_GUIDE.md) — Full compression pipeline overview
- [COMPRESSION_ENGINES.md](./COMPRESSION_ENGINES.md) — Engine registry and built-in engines
- [EXTENDING_COMPRESSION.md](./EXTENDING_COMPRESSION.md) — Custom engines, language packs, stacked pipelines
- Source: `open-sse/services/compression/engines/rtk/` (63 files, ~70KB)

75
Release-Checklist.md

@@ -109,7 +109,7 @@ Breaking changes: add `BREAKING CHANGE:` footer or `!` after the scope (e.g. `fe
- [ ] `npm run i18n:check` exits 0 — translation state (`.i18n-state.json`) in sync with source docs (no drifted sources in strict mode; warn-mode advisory is acceptable for last-minute doc touch-ups, but should be 0 before tagging)
- [ ] `npm run i18n:check-ui-coverage` exits 0 — every UI locale at or above the 80% coverage floor
- [ ] `npm run i18n:sync-ui:dry` reports 0 missing keys across all 40 locales
- [ ] `npm run i18n:sync-ui:dry` reports 0 missing keys across all 42 locales
- [ ] If source English docs changed, run `npm run i18n:run` (requires `OMNIROUTE_TRANSLATION_API_KEY` in `.env`) before tagging
- [ ] Translation contributions can be deferred to next release if minor (track in CHANGELOG)
@@ -144,11 +144,38 @@ If `electron/` changed:
- [ ] `electron/package.json` version matches root `package.json`
- [ ] Auto-update channel pointer updated if releasing to `stable`
### Build Layout
The repository uses three distinct output directories — never mix them up:
| Directory | Purpose | Tracked? |
| ------------- | ------------------------------------------------------------- | -------- |
| `src/` | Application source (TypeScript / TSX) | Yes |
| `.build/` | Build intermediates — `next build` output (`distDir`) | No (gitignored) |
| `dist/` | Shippable npm bundle — assembled by `assembleStandalone` | No (gitignored) |
> **Operator note:** the remote VPS image directory remains `/usr/lib/node_modules/omniroute/app/`.
> Only the **in-repo** build output moved (`app/` → `dist/`). The deploy skills rsync
> `dist/` contents into the remote `app/` dir — no VPS path changes required.
**Single-build flow:**
```
npm run build:release
└─ rm -rf .build dist (clean)
└─ next build → .build/next/ (intermediates)
└─ assembleStandalone (copies standalone + static + public + natives → dist/)
└─ writes dist/BUILD_SHA (HEAD sentinel)
```
Do NOT run `npm run build` followed by a separate `npm run build:cli` for deploy — use
`npm run build:release` which does a clean rebuild + sentinel in one command.
### Artifact Validation
- [ ] `npm run build:cli` succeeds
- [ ] `npm run build:release` succeeds and `dist/BUILD_SHA` == `git rev-parse --short HEAD`
- [ ] `npm run check:pack-artifact` clean — no `app.__qa_backup`, `scripts/scratch`, `package-lock.json`, or other local residue
- [ ] `npm run build` produces a working standalone Next.js bundle
- [ ] `dist/server.js` exists after build
### Tagging & Release
@@ -166,10 +193,14 @@ If `electron/` changed:
### Deploy
Deploy skills use the light rsync flow — no `npm pack`, no `npm i -g`:
- [ ] Use deploy skill that matches target:
- `/deploy-vps-local-cc` — local VPS (192.168.0.15)
- `/deploy-vps-akamai-cc` — Akamai VPS (69.164.221.35)
- `/deploy-vps-both-cc` — both
- [ ] Before deploying, confirm `dist/BUILD_SHA` == `git rev-parse --short HEAD`
- [ ] Build must run where `node_modules` is real (main checkout or `npm ci`'d worktree — NOT a symlinked worktree)
- [ ] Smoke test deployed instance:
- Open `/dashboard/health` → check version string matches release
- Run a `/v1/chat/completions` request against a known provider
@@ -185,6 +216,44 @@ If `electron/` changed:
- [ ] Open milestone for next version
- [ ] If critical: pin discussion or post in `news.json` for in-app banner
## Embedded Services smoke (v3.8.4+)
Before shipping any release that includes embedded services changes, verify:
### Fresh-DB boot (catches migration collisions — added after v3.8.4 hotfix)
- [ ] `DATA_DIR=$(mktemp -d) npm start &` — wait 10 s for boot
- [ ] `curl -s http://127.0.0.1:20128/api/services/9router/status | jq '.tool'` returns `"9router"` (NOT 404, NOT 500). Confirms migration `071_services.sql` applied + row seeded.
- [ ] `sqlite3 $DATA_DIR/storage.sqlite "PRAGMA table_info(version_manager);" | grep -E "provider_expose|logs_buffer_path|last_sync_at"` returns 3 rows.
- [ ] `sqlite3 $DATA_DIR/storage.sqlite "PRAGMA table_info(webhooks);" | grep -E "kind|metadata_encrypted"` returns 2 rows (validates `070_webhooks_kind_metadata.sql` applied).
- [ ] `node --import tsx/esm --test tests/unit/db/no-migration-collisions.test.ts` passes — guards against future collisions.
### 9Router
- [ ] `POST /api/services/9router/install` returns 200 with `installedVersion` in under 2 min
- [ ] `POST /api/services/9router/start` returns 200 and `state: "running"` in under 30 s
- [ ] `GET /api/services/9router/status` reports `health: "healthy"`
- [ ] `POST /v1/chat/completions` with `"model": "9router/auto/..."` returns 200 (end-to-end routing through 9Router)
- [ ] `GET /dashboard/providers/services/9router/embed/dashboard` renders the 9Router native UI inside the proxy (no direct `127.0.0.1:port` iframe)
- [ ] `POST /api/services/9router/rotate-key` returns `{ keyRotated: true }` and service restarts cleanly
- [ ] `POST /api/services/9router/stop` returns 200 and `state: "stopped"`
- [ ] `GET /api/services/9router/logs?tail=50` returns SSE stream with `snapshot` event containing recent lines
- [ ] Install in environment without `npm` in PATH returns 500 with a friendly (non-stack-trace) error message
### CLIProxyAPI
- [ ] `POST /api/services/cliproxy/install` returns 200 in under 2 min
- [ ] `POST /api/services/cliproxy/start` returns 200 and `state: "running"` in under 30 s
- [ ] `GET /api/services/cliproxy/status` reports `health: "healthy"`
- [ ] `POST /api/services/cliproxy/stop` returns 200 and `state: "stopped"`
- [ ] `GET /api/services/cliproxy/logs?tail=50` returns SSE stream
### Security regression
- [ ] `curl -H "X-Forwarded-For: 1.2.3.4" http://localhost:20128/api/services/9router/start` returns `403 LOCAL_ONLY`
- [ ] `curl -H "X-Forwarded-For: 1.2.3.4" http://localhost:20128/api/services/cliproxy/start` returns `403 LOCAL_ONLY`
- [ ] Error responses from `/api/services/*` do not contain `err.stack` or absolute file paths
## v3.8.0+ checks
Before shipping any v3.8.x release, verify these additional items:

280
Repository-Map.md

@@ -4,7 +4,7 @@
# Repository Map
> **One-line description for every directory and root file.**
> Last updated: 2026-05-13 — OmniRoute v3.8.0
> Last updated: 2026-06-15 — OmniRoute v3.8.26
>
> Use this map to navigate the codebase quickly. For deep dives, follow links to dedicated docs.
@@ -20,8 +20,13 @@ OmniRoute/
├── docs/ # Public documentation (you are here)
├── tests/ # All test suites (unit, integration, e2e, protocols-e2e)
├── public/ # Next.js static assets, PWA manifest, service worker, icons
├── config/ # Static config files
├── config/ # Static config + quality-gate state (i18n, payloadRules, quality/)
├── images/ # Marketing / README image assets
├── @omniroute/ # Publishable companion packages (opencode-plugin, opencode-provider)
├── skills/ # CLI/agent skill packs (cli-* + omni-* + config-codex-cli)
├── examples/ # Sample plugins + omniroute-cmd-hello starter
├── contrib/ # Community contributions (podman/)
├── .source/ # Fumadocs source config (source.config.mjs + server/browser/dynamic)
├── .github/ # GitHub Actions workflows + issue templates + PR template
├── .husky/ # Git hooks (pre-commit, pre-push)
├── .claude/ # Claude Code slash commands (project-scoped)
@@ -31,6 +36,7 @@ OmniRoute/
├── _mono_repo/ # Historic subprojects (cloud, site, vscode-extension)
├── _references/ # Read-only reference clones from related OSS projects
├── _tasks/ # Per-release task tracking files (informal)
├── .build/ .worktrees/ dist/ # local build / git-worktree / build-output scratch (gitignored)
├── .issues/ # Local issue cache (gitignored)
├── .playwright-mcp/ # Playwright MCP test artifacts
├── coverage/ # c8 coverage output (gitignored)
@@ -45,45 +51,63 @@ OmniRoute/
## Root files
| File | Purpose |
| ------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| **README.md** | Marketing landing page + quick start + feature matrix (see also `llm.txt`) |
| **CHANGELOG.md** | Per-release changelog (auto-generated by `/version-bump-cc` skill) |
| **LICENSE** | MIT license text |
| **CLAUDE.md** | Project rules for Claude Code agents (hard rules, conventions, scenarios) |
| **AGENTS.md** | Same as CLAUDE.md but for non-Claude AI agents (Codex, Cursor, etc.) |
| **GEMINI.md** | Concise rules for Gemini-based agents (subset of CLAUDE.md) |
| **CONTRIBUTING.md** | Contributor guide: setup, conventional commits, testing, PR flow |
| **SECURITY.md** | Vulnerability reporting policy, supported versions, threat model |
| **CODE_OF_CONDUCT.md** | Contributor Covenant — community behavior expectations |
| **llm.txt** | Plain-text landing optimized for LLM crawlers (SEO for AI assistants) |
| **Tuto_Qdrant.md** | Tutorial for enabling Qdrant vector memory — **integration currently dormant** (see banner; primary memory docs in `docs/frameworks/MEMORY.md`) |
| **package.json** | npm manifest, scripts, dependencies, engines, c8 coverage gate |
| **package-lock.json** | Locked dependency tree |
| **tsconfig.json** | Root TypeScript config |
| **tsconfig.typecheck-core.json** | Typecheck config for `src/` core |
| **tsconfig.typecheck-noimplicit-core.json** | Strict (`noImplicitAny`) typecheck |
| **tsconfig.tsbuildinfo** | TS incremental build cache (gitignored) |
| **next.config.mjs** | Next.js 16 build configuration (standalone output) |
| **next-env.d.ts** | Next.js auto-generated env types |
| **eslint.config.mjs** | ESLint flat config (rules per project area) |
| **prettier.config.mjs** | Prettier formatting rules |
| **postcss.config.mjs** | PostCSS config for Tailwind/CSS pipeline |
| **playwright.config.ts** | Playwright E2E test config |
| **vitest.config.ts** | Vitest config (default suite) |
| **vitest.mcp.config.ts** | Vitest config for MCP server / autoCombo / cache suites |
| **sonar-project.properties** | SonarQube/SonarCloud config (code quality) |
| **Dockerfile** | Multi-stage Docker build (builder → runner-base → runner-cli) |
| **docker-compose.yml** | Dev compose with 4 profiles (base, cli, host, cliproxyapi) + redis sidecar |
| **docker-compose.prod.yml** | Production compose (port 20130, redis, named volumes) |
| **.dockerignore** | Files excluded from Docker context |
| **fly.toml** | Fly.io deployment config (region `sin`, port 20128, /data volume) |
| **.env.example** | Template env file (815 lines, auto-copied to `.env` on first install) |
| **.gitignore** | Git ignore patterns |
| **.npmignore** | npm publish exclusion list |
| **.npmrc** | npm config (registry, lockfile policy) |
| **.node-version** | Node version pin (used by nvm-compatible tools) |
| **.nvmrc** | Node version pin for nvm |
| File | Purpose |
| ------------------------------------------- | ---------------------------------------------------------------------------------------- |
| **README.md** | Marketing landing page + quick start + feature matrix (see also `llm.txt`) |
| **CHANGELOG.md** | Per-release changelog (auto-generated by `/version-bump-cc` skill) |
| **LICENSE** | MIT license text |
| **CLAUDE.md** | Project rules for Claude Code agents (hard rules, conventions, scenarios) |
| **AGENTS.md** | Same as CLAUDE.md but for non-Claude AI agents (Codex, Cursor, etc.) |
| **GEMINI.md** | Concise rules for Gemini-based agents (subset of CLAUDE.md) |
| **CONTRIBUTING.md** | Contributor guide: setup, conventional commits, testing, PR flow |
| **SECURITY.md** | Vulnerability reporting policy, supported versions, threat model |
| **CODE_OF_CONDUCT.md** | Contributor Covenant — community behavior expectations |
| **llm.txt** | Plain-text landing optimized for LLM crawlers (SEO for AI assistants) |
| **package.json** | npm manifest, scripts, dependencies, engines, c8 coverage gate |
| **package-lock.json** | Locked dependency tree |
| **tsconfig.json** | Root TypeScript config |
| **tsconfig.typecheck-core.json** | Typecheck config for `src/` core |
| **tsconfig.typecheck-noimplicit-core.json** | Strict (`noImplicitAny`) typecheck |
| **tsconfig.tsbuildinfo** | TS incremental build cache (gitignored) |
| **next.config.mjs** | Next.js 16 build configuration (standalone output) |
| **next-env.d.ts** | Next.js auto-generated env types |
| **eslint.config.mjs** | ESLint flat config (rules per project area) |
| **prettier.config.mjs** | Prettier formatting rules |
| **postcss.config.mjs** | PostCSS config for Tailwind/CSS pipeline |
| **playwright.config.ts** | Playwright E2E test config |
| **vitest.config.ts** | Vitest config (default suite) |
| **vitest.mcp.config.ts** | Vitest config for MCP server / autoCombo / cache suites |
| **sonar-project.properties** | SonarQube/SonarCloud config (code quality) |
| **Dockerfile** | Multi-stage Docker build (builder → runner-base → runner-cli) |
| **docker-compose.yml** | Dev compose with 4 profiles (base, cli, host, cliproxyapi) + redis sidecar |
| **docker-compose.prod.yml** | Production compose (port 20130, redis, named volumes) |
| **.dockerignore** | Files excluded from Docker context |
| **fly.toml** | Fly.io deployment config (region `sin`, port 20128, /data volume) |
| **.env.example** | Template env file (auto-copied to `.env` on first install) |
| **.gitignore** | Git ignore patterns |
| **.npmignore** | npm publish exclusion list |
| **.npmrc** | npm config (registry, lockfile policy) |
| **.node-version** | Node version pin (used by nvm-compatible tools) |
| **.nvmrc** | Node version pin for nvm |
| **eslint.complexity.config.mjs** | ESLint config for the complexity ratchet (`scripts/check/check-complexity.mjs --config`) |
| **eslint.sonarjs.config.mjs** | ESLint config for SonarJS rules (cognitive complexity / duplication) |
| **source.config.ts** | Fumadocs `defineDocs` source config (feeds `.source/`) |
| **knip.json** | Knip config — unused files/exports/deps (feeds the dead-code gate) |
| **stryker.conf.json** | Stryker mutation-testing config |
| **.size-limit.json** | size-limit bundle budget config |
| **semcheck.yaml** | semcheck (spec↔code drift) config |
| **promptfooconfig.yaml** | promptfoo eval config |
| **.gitleaks.toml** | gitleaks secret-scan ruleset |
| **.zizmor.yml** | zizmor GitHub-Actions security-lint config |
| **socket.yml** | Socket.dev supply-chain config |
| **news.json** | In-app release-notes feed (read by `src/shared/utils/releaseNotes.ts`) |
| **flake.nix** / **flake.lock** | Nix dev-shell definition + lock |
| **.env** | Local secrets (gitignored — generated from `.env.example`) |
> **Moved out of the root in v3.8.26 (declutter):**
>
> - **→ `config/quality/`:** `quality-baseline.json`, `complexity-baseline.json`, `duplication-baseline.json`, `file-size-baseline.json`, `test-discovery-baseline.json`, `dependency-allowlist.json`, `.license-allowlist.json`, and the generated `quality-metrics.json` (gitignored). See [`## config/`](#config--static-configs--quality-gate-state).
> - **→ `docs/ops/`:** `DOCUMENTATION_AUDIT_REPORT.md`.
---
@@ -98,7 +122,9 @@ src/
├── shared/ # Shared between server and client where safe (constants, types, validation, contracts, utils)
├── i18n/ # next-intl config + per-locale message JSON (30+ locales)
├── middleware/ # Next.js middleware (request enrichment, locale detection)
├── mitm/ # MITM proxy helpers (Linux cert install, antigravity stealth)
├── mitm/ # MITM proxy core: cert gen/install, handlers, targets, inspector, masks, passthrough
│ ├── handlers/ # 9 IDE-agent handler classes extending MitmHandlerBase (antigravity, kiro, copilot, codex, cursor, zed, claudeCode, openCode, trae)
│ └── inspector/ # Traffic capture layer: buffer (in-memory ring), sseMerger, conversationNormalizer, kindDetector, contextKey, httpProxyServer, systemProxyConfig
├── models/ # Model adapter glue (legacy shim)
├── scripts/ # In-tree maintenance scripts (e.g., backfillAggregation)
├── sse/ # Legacy SSE handlers/services (chat.ts, chatHelpers.ts, services/auth.ts)
@@ -112,65 +138,84 @@ src/
### `src/app/` — App Router (Next.js 16)
| Path | Purpose |
| ---------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `app/api/v1/` | Public OpenAI-compat API (~25 sub-routes: chat, completions, embeddings, files, batches, audio, images, videos, music, rerank, moderations, search, ws, agents, accounts, providers, etc.) |
| `app/api/v1beta/` | Gemini-style API endpoints |
| `app/api/` (non-v1) | Management/admin routes (~60 directories: providers, combos, settings, mcp, a2a, evals, memory, skills, webhooks, compliance, resilience, monitoring, tunnels, cli-tools, etc.) |
| `app/a2a/` | A2A JSON-RPC 2.0 entry point (`POST /a2a`) |
| `app/.well-known/agent.json/` | A2A Agent Card (discovery) |
| `app/(dashboard)/dashboard/` | Dashboard UI pages (~30 pages: providers, combos, settings, memory, skills, webhooks, evals, audit, batch, cache, costs, health, system, etc.) |
| `app/docs/` | Embedded documentation viewer (renders `docs/*.md`) |
| `app/landing/` | Marketing landing page |
| `app/login/`, `forgot-password/`, `forbidden/` | Auth-related pages |
| `app/{400,401,403,408,429,500,502,503}/` | HTTP error pages |
| `app/maintenance/`, `offline/`, `status/`, `privacy/`, `terms/`, `callback/` | Static/status pages |
| `app/layout.tsx`, `page.tsx`, `manifest.ts`, `globals.css` | Root layout, home, PWA manifest, global CSS |
| `app/error.tsx`, `global-error.tsx`, `not-found.tsx`, `loading.tsx` | Error boundaries |
| Path | Purpose |
| ---------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `app/api/v1/` | Public OpenAI-compat API (~25 sub-routes: chat, completions, embeddings, files, batches, audio, images, videos, music, rerank, moderations, search, ws, agents, accounts, providers, etc.) |
| `app/api/v1beta/` | Gemini-style API endpoints |
| `app/api/playground/` | Playground Studio routes: `improve-prompt/` (POST — LLM prompt rewriter), `presets/` (GET list / POST create), `presets/[id]/` (GET / PUT / DELETE) — see `docs/frameworks/PLAYGROUND_STUDIO.md` |
| `app/api/` (non-v1) | Management/admin routes (~60 directories: providers, combos, settings, mcp, a2a, evals, memory, skills, webhooks, compliance, resilience, monitoring, tunnels, cli-tools, etc.) |
| `app/api/tools/agent-bridge/` | AgentBridge REST API — 12 routes (server control, agent state/DNS/mappings, bypass, cert, upstream-CA). LOCAL_ONLY + SPAWN_CAPABLE. See `docs/frameworks/AGENTBRIDGE.md §7`. |
| `app/api/tools/traffic-inspector/` | Traffic Inspector REST + WS API — 16+ routes (requests, sessions, hosts, capture-modes, export, ws). LOCAL_ONLY + SPAWN_CAPABLE. See `docs/frameworks/TRAFFIC_INSPECTOR.md §8`. |
| `app/a2a/` | A2A JSON-RPC 2.0 entry point (`POST /a2a`) |
| `app/.well-known/agent.json/` | A2A Agent Card (discovery) |
| `app/(dashboard)/dashboard/` | Dashboard UI pages (~35 pages: providers, combos, settings, memory, skills, webhooks, evals, audit, batch, cache, costs, health, system, activity, etc.) |
| `app/(dashboard)/dashboard/search-tools/` | Search Tools Studio UI (3 tabs: Search/Scrape/Compare + SearchConceptCard + ProviderCatalog) — see `docs/frameworks/SEARCH_TOOLS_STUDIO.md` |
| `app/(dashboard)/dashboard/` | Dashboard UI pages (~30 pages: providers, combos, settings, memory, skills, webhooks, evals, audit, batch, cache, costs, health, system, etc.) |
| `app/(dashboard)/dashboard/memory/` | Memory Studio (plan 21): `page.tsx` (3-tab shell), `components/` (MemoryConceptCard, MemoryEngineStatus, EmbeddingSourceSelector, EditMemoryModal, RetrievePreview, QdrantConfigCard, RerankConfigCard), `components/tabs/` (MemoriesTab, PlaygroundTab, EngineTab), `hooks/` (useEngineStatus, useMemorySettings) |
| `app/(dashboard)/dashboard/tools/agent-bridge/` | AgentBridge dashboard page — server card, 9 agent cards, setup wizard, model mapping, bypass list. i18n PT-BR + EN. See `docs/frameworks/AGENTBRIDGE.md`. |
| `app/(dashboard)/dashboard/tools/traffic-inspector/` | Traffic Inspector dashboard page — DevTools split, 7 detail tabs, 4 capture mode toggles, session recorder, context colorization. i18n PT-BR + EN. See `docs/frameworks/TRAFFIC_INSPECTOR.md`. |
| `app/(dashboard)/dashboard/activity/` | Activity feed page (Group B): `page.tsx` (server) + `ActivityFeedClient.tsx` + `components/{ActivityFeed,ActivityItem,DayHeader,EventTypeFilter}.tsx` — see `docs/architecture/MONITORING_SECTIONS.md` |
| `app/(dashboard)/dashboard/costs/quota-share/` | Quota Sharing page (Group B): `QuotaSharePageClient.tsx` + `components/{PoolCard,DimensionBar,AllocationTable,BurnRateChart,QuotaConceptCard,CreatePoolModal,EditAllocationsModal}.tsx` + `hooks/{usePools,usePoolUsage,useLocalStoragePoolMigration}.ts` |
| `app/(dashboard)/dashboard/costs/quota-share/plans/` | Provider plan config page (Group B): `page.tsx` + `ProviderPlanConfigClient.tsx` — quota dimensions per connection override |
| `app/docs/` | Embedded documentation viewer (renders `docs/*.md`) |
| `app/landing/` | Marketing landing page |
| `app/login/`, `forgot-password/`, `forbidden/` | Auth-related pages |
| `app/{400,401,403,408,429,500,502,503}/` | HTTP error pages |
| `app/maintenance/`, `offline/`, `status/`, `privacy/`, `terms/`, `callback/` | Static/status pages |
| `app/layout.tsx`, `page.tsx`, `manifest.ts`, `globals.css` | Root layout, home, PWA manifest, global CSS |
| `app/error.tsx`, `global-error.tsx`, `not-found.tsx`, `loading.tsx` | Error boundaries |
### `src/lib/` — Core libraries (~50 modules)
| Module | Purpose |
| ---------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `a2a/` | A2A protocol task manager, skills (5), streaming |
| `acp/` | CLI Agent Registry (local CLI discovery — see `docs/frameworks/AGENT_PROTOCOLS_GUIDE.md`) |
| `api/` | Shared API helpers (`requireManagementAuth`, validation) |
| `auth/` | Session, password hashing, token validation |
| `batches/` | OpenAI Batches API handlers |
| `catalog/` | Provider catalog Zod validation + capability resolution |
| `cloudAgent/` | Cloud Agents (Codex Cloud, Devin, Jules) — see `docs/frameworks/CLOUD_AGENT.md` |
| `combos/` | Combo resolution + reorder helpers |
| `compliance/` | Audit log + provider audit — see `docs/security/COMPLIANCE.md` |
| `compression/` | Compression engine glue (engines live in `open-sse/services/compression/`) |
| `config/` | Runtime config helpers |
| `db/` | 45+ domain DB modules + 55 migrations (always go through here for SQLite) |
| `display/` | UI formatting helpers (cost, latency, etc.) |
| `embeddings/` | Embeddings service helpers |
| `env/` | Env variable parsing + validation |
| `evals/` | Eval framework (suites, runner, runtime) — see `docs/frameworks/EVALS.md` |
| `guardrails/` | PII masker, prompt injection, vision bridge — see `docs/security/GUARDRAILS.md` |
| `jobs/` | Background jobs (cron-like) |
| `memory/` | Conversational memory (SQLite FTS5 + Qdrant) — see `docs/frameworks/MEMORY.md` |
| `monitoring/` | Health checks, metrics emission |
| `oauth/` | OAuth flows for 14 providers (claude, codex, antigravity, cursor, github, gemini, kimi-coding, kilocode, cline, qwen, kiro, qoder, gitlab-duo, windsurf) |
| `plugins/` | Plugin registry |
| `promptCache/` | Anthropic-style prompt cache breakpoints |
| `skills/` | Skills framework (built-in + marketplace + SkillsSH) — see `docs/frameworks/SKILLS.md` |
| `webhookDispatcher.ts` | HMAC webhook delivery — see `docs/frameworks/WEBHOOKS.md` |
| `cloudflaredTunnel.ts`, `ngrokTunnel.ts` | Tunnel managers — see `docs/ops/TUNNELS_GUIDE.md` |
| `oneproxySync.ts`, `oneproxyRotator.ts` | 1proxy free proxy marketplace — see `docs/ops/PROXY_GUIDE.md` |
| `cloudSync.ts`, `initCloudSync.ts` | Optional cloud sync of state |
| `localDb.ts` | Re-export barrel for db modules (no logic — re-exports only) |
| `cacheLayer.ts`, `idempotencyLayer.ts` | Request caching + idempotency |
| (~30 more top-level files) | Specialized helpers (logEnv, modelsDevSync, piiSanitizer, etc.) |
| Module | Purpose |
| ---------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `a2a/` | A2A protocol task manager, skills (5), streaming |
| `acp/` | CLI Agent Registry (local CLI discovery — see `docs/frameworks/AGENT_PROTOCOLS_GUIDE.md`) |
| `api/` | Shared API helpers (`requireManagementAuth`, validation) |
| `auth/` | Session, password hashing, token validation |
| `batches/` | OpenAI Batches API handlers |
| `catalog/` | Provider catalog Zod validation + capability resolution |
| `cloudAgent/` | Cloud Agents (Codex Cloud, Devin, Jules) — see `docs/frameworks/CLOUD_AGENT.md` |
| `combos/` | Combo resolution + reorder helpers |
| `audit/` | Activity feed helpers: `highLevelActions.ts` (allowlist + `isHighLevelAction()`), `activityIcons.ts` (action → icon/verb map), `timeline.ts` (groupByDay/relativeTime) — see `docs/architecture/MONITORING_SECTIONS.md` |
| `compliance/` | Audit log + provider audit — see `docs/security/COMPLIANCE.md` |
| `compression/` | Compression engine glue (engines live in `open-sse/services/compression/`) |
| `config/` | Runtime config helpers |
| `db/` | 45+ domain DB modules + 55 migrations (always go through here for SQLite) |
| `quota/` | Quota Sharing Engine: `dimensions.ts` (types/Zod), `types.ts` (QuotaStore interface), `sqliteQuotaStore.ts`, `redisQuotaStore.ts`, `storeFactory.ts`, `fairShare.ts`, `burnRate.ts`, `planResolver.ts`, `planRegistry.ts`, `saturationSignals.ts`, `enforce.ts`, `spendRecorder.ts` — see `docs/routing/QUOTA_SHARE.md` |
| `display/` | UI formatting helpers (cost, latency, etc.) |
| `embeddings/` | Embeddings service helpers |
| `env/` | Env variable parsing + validation |
| `evals/` | Eval framework (suites, runner, runtime) — see `docs/frameworks/EVALS.md` |
| `guardrails/` | PII masker, prompt injection, vision bridge — see `docs/security/GUARDRAILS.md` |
| `jobs/` | Background jobs (cron-like) |
| `memory/` | Conversational memory (SQLite FTS5 + sqlite-vec hybrid RRF + Qdrant tier 2) — see `docs/frameworks/MEMORY.md` |
| `memory/embedding/` | Multi-source embedding layer: `index.ts` (resolver), `remote.ts`, `staticPotion.ts`, `transformersLocal.ts`, `cache.ts`, `types.ts` (plan 21) |
| `memory/vectorStore.ts` | sqlite-vec v0.1.9 wrapper — KNN brute-force + hybrid RRF (FTS5 + vector, k=60). Lazy-init, degrades gracefully when sqlite-vec unavailable. (plan 21) |
| `memory/reindex.ts` | `runReindexBatch()` — processes memories with `needs_reindex=1` in background; called by `POST /api/memory/reindex` and lazy-backfill path. (plan 21) |
| `monitoring/` | Health checks, metrics emission |
| `oauth/` | OAuth flows for 14 providers (claude, codex, antigravity, cursor, github, gemini, kimi-coding, kilocode, cline, qwen, kiro, qoder, gitlab-duo, windsurf) |
| `plugins/` | Plugin registry |
| `promptCache/` | Anthropic-style prompt cache breakpoints |
| `skills/` | Skills framework (built-in + marketplace + SkillsSH) — see `docs/frameworks/SKILLS.md` |
| `playground/` | Playground Studio shared helpers: `codeExport.ts` (curl/Python/TS generator), `promptImprover.ts` (meta-prompt builder), `streamMetrics.ts` (pure TTFT/TPS), `types.ts` (pricing table) — see `docs/frameworks/PLAYGROUND_STUDIO.md` |
| `webhookDispatcher.ts` | HMAC webhook delivery — see `docs/frameworks/WEBHOOKS.md` |
| `cloudflaredTunnel.ts`, `ngrokTunnel.ts` | Tunnel managers — see `docs/ops/TUNNELS_GUIDE.md` |
| `oneproxySync.ts`, `oneproxyRotator.ts` | 1proxy free proxy marketplace — see `docs/ops/PROXY_GUIDE.md` |
| `cloudSync.ts`, `initCloudSync.ts` | Optional cloud sync of state |
| `localDb.ts` | Re-export barrel for db modules (no logic — re-exports only) |
| `cacheLayer.ts`, `idempotencyLayer.ts` | Request caching + idempotency |
| (~30 more top-level files) | Specialized helpers (logEnv, modelsDevSync, piiSanitizer, etc.) |
### `src/db/` — Database (45+ modules + 55 migrations)
| Subdir | Purpose |
| ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `db/core.ts` | `getDbInstance()` singleton with WAL journaling |
| `db/migrations/` | 55 versioned SQL files (idempotent, transactional, numbered `001`..`055`) |
| `db/<domain>.ts` | One module per domain: providers, combos, apiKeys, users, sessions, usage, audit*log, webhooks, skills, memory_entries, cloud_agent_tasks, evals*\*, reasoning_cache, etc. |
| Subdir | Purpose |
| ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `db/core.ts` | `getDbInstance()` singleton with WAL journaling |
| `db/migrations/` | Versioned SQL files (idempotent, transactional). `073_memory_vec.sql` adds `memory_vec_meta` + `needs_reindex` column (plan 21). |
| `db/playgroundPresets.ts` | CRUD module for Playground Studio presets (`listPlaygroundPresets`, `getPlaygroundPreset`, `createPlaygroundPreset`, `updatePlaygroundPreset`, `deletePlaygroundPreset`) |
| `db/memoryVec.ts` | CRUD for `memory_vec_meta` (active_dim, embedding_signature, last_reset_at, vec_loaded) + `markMemoryNeedsReindex`, `getMemoryReindexQueue`, etc. (plan 21) |
| `db/<domain>.ts` | One module per domain: providers, combos, apiKeys, users, sessions, usage, audit*log, webhooks, skills, memory_entries, cloud_agent_tasks, evals*\*, reasoning_cache, etc. |
### `src/domain/`
@@ -199,7 +244,7 @@ src/
| -------------------------------- | ---------------------------------------------------------------------- |
| `constants/providers.ts` | **226 providers** with Zod validation (source of truth) |
| `constants/cliTools.ts` | External CLI tool registry |
| `constants/routingStrategies.ts` | **15 routing strategies** with priorities |
| `constants/routingStrategies.ts` | **14 routing strategies** with priorities |
| `constants/publicApiRoutes.ts` | Routes that require Bearer (vs management) auth |
| `constants/upstreamHeaders.ts` | Header denylist for upstream requests |
| `validation/schemas.ts` | ~80 Zod schemas (single source of truth for API contracts) |
@@ -223,7 +268,7 @@ open-sse/
├── translator/ # Format converters (9 request, 8 response, 9 helpers)
├── transformer/ # Responses API ↔ Chat Completions (TransformStream)
├── services/ # ~80+ service modules (combo, accountFallback, autoCombo, reasoningCache, claude code/chatgpt stealth, modelDeprecation, taskAwareRouter, workflowFSM, etc.)
├── mcp-server/ # MCP server (87 tools, 3 transports, ~13 scopes)
├── mcp-server/ # MCP server (87 tools, 3 transports, 30 scopes)
├── config/ # Provider/model registries, header config, model aliases
├── utils/ # TLS client, proxy fetch/dispatcher, network helpers
├── index.ts # Workspace entry
@@ -360,7 +405,7 @@ open-sse/
| Doc | Purpose |
| -------------------------- | ------------------------------------------------------------------- |
| `MCP-SERVER.md` | MCP server: 87 tools, 3 transports, ~13 scopes, REST endpoints |
| `MCP-SERVER.md` | MCP server: 87 tools, 3 transports, 30 scopes, REST endpoints |
| `A2A-SERVER.md` | A2A v0.3: JSON-RPC, 5 skills, REST helpers, agent card |
| `AGENT_PROTOCOLS_GUIDE.md` | Unified guide: A2A vs ACP vs Cloud Agents |
| `CLOUD_AGENT.md` | Codex Cloud / Devin / Jules orchestration |
@@ -404,7 +449,7 @@ open-sse/
| Subdir | Purpose |
| ------------------------- | ------------------------------------------------------------------------------------- |
| `docs/archive/` | Archived/historical docs (e.g., `RFC-AUTO-ASSESSMENT-DRAFT.md` — superseded by EVALS) |
| `docs/i18n/` | Localized doc translations (~40 locales) |
| `docs/i18n/` | Localized doc translations (~42 locales) |
| `docs/screenshots/` | Image assets for guides |
| `docs/superpowers/plans/` | Implementation plans (generated by `superpowers:writing-plans` skill) |
@@ -431,9 +476,24 @@ open-sse/
---
## `config/` — Static Configs
## `config/` — Static Configs + Quality-Gate State
Shipped configuration templates and sample files (referenced by setup wizard).
Shipped configuration templates plus the committed quality-gate baselines
(moved here from the repo root in v3.8.26 to keep the root lean).
| Path | Purpose |
| --------------------------------------------- | -------------------------------------------------------------------------------- |
| `config/i18n.json` | Locale list + metadata (canonical source for the 42-locale count) |
| `config/i18n-schema.json` | JSON schema validating `i18n.json` |
| `config/payloadRules.json` | Upstream payload sanitization rules |
| `config/quality/quality-baseline.json` | Multi-metric ratchet baseline (`scripts/quality/check-quality-ratchet.mjs`) |
| `config/quality/complexity-baseline.json` | Frozen ESLint-complexity baseline (`check-complexity.mjs`) |
| `config/quality/duplication-baseline.json` | Frozen jscpd duplication baseline (`check-duplication.mjs`) |
| `config/quality/file-size-baseline.json` | Frozen per-file size baseline (`check-file-size.mjs`) |
| `config/quality/test-discovery-baseline.json` | Frozen orphan-test baseline (`check-test-discovery.mjs`) |
| `config/quality/dependency-allowlist.json` | Approved dependencies allowlist (`check-deps.mjs`) |
| `config/quality/.license-allowlist.json` | SPDX license allowlist (`check-licenses.mjs`) |
| `config/quality/quality-metrics.json` | Ephemeral collected metrics (generated by `collect-metrics.mjs`; **gitignored**) |
---
@@ -460,15 +520,15 @@ Shipped configuration templates and sample files (referenced by setup wizard).
## `.claude/` — Claude Code Slash Commands
| File | Purpose |
| ----------------------------------------------------------------- | -------------------------------------------------- |
| `commands/version-bump-cc.md` | `/version-bump-cc` — bump version + auto-changelog |
| `commands/generate-release-cc.md` | `/generate-release-cc` — full release workflow |
| `commands/deploy-vps-{local,akamai,both}-cc.md` | Deploy to VPS |
| `commands/capture-release-evidences-cc.md` | Browser-record new features as WebP |
| `commands/review-{prs,discussions}-cc.md` | Triage GitHub PRs/discussions |
| `commands/{issue-triage,resolve-issues,implement-features}-cc.md` | Issue workflows |
| `settings.local.json` | Per-project Claude Code settings |
| File | Purpose |
| --------------------------------------------------- | -------------------------------------------------- |
| `commands/version-bump-cc.md` | `/version-bump-cc` — bump version + auto-changelog |
| `commands/generate-release-cc.md` | `/generate-release-cc` — full release workflow |
| `commands/deploy-vps-{local,akamai,both}-cc.md` | Deploy to VPS |
| `commands/capture-release-evidences-cc.md` | Browser-record new features as WebP |
| `commands/review-{prs,discussions}-cc.md` | Triage GitHub PRs/discussions |
| `commands/{review-issues,implement-features}-cc.md` | Issue workflows |
| `settings.local.json` | Per-project Claude Code settings |
---

34
Resilience-Guide.md

@@ -27,16 +27,19 @@ OmniRoute has three distinct but related resilience mechanisms. Each has a diffe
**States:**
- `CLOSED` — normal traffic allowed
- `DEGRADED` — traffic still allowed, but elevated provider failures are being tracked
- `OPEN` — provider temporarily blocked; combo routing skips it
- `HALF_OPEN` — reset timeout elapsed; probe request allowed
**Defaults (`open-sse/config/constants.ts`):**
**Configurable defaults (`open-sse/config/constants.ts`, exposed in Dashboard → Settings → Resilience):**
| Class | Threshold | Reset timeout |
| ------- | ---------- | ------------- |
| OAuth | 3 failures | 60s |
| API-key | 5 failures | 30s |
| Local | 2 failures | 15s |
| Class | Degraded at | Opens at | Reset timeout |
| ------- | ----------- | ----------- | ------------- |
| OAuth | 5 failures | 8 failures | 60s |
| API-key | 7 failures | 12 failures | 30s |
| Local | derived | 2 failures | 15s |
`degradationThreshold` controls when a provider enters `DEGRADED`; `failureThreshold` controls when it opens and is skipped. Local provider profiles are not exposed on the Resilience settings page yet.
**Trip codes:** only provider-level statuses `[408, 500, 502, 503, 504]`. Do NOT trip for account-level errors (most 401/403/429 — those belong to cooldown or lockout).
@@ -114,10 +117,11 @@ Lists active lockouts with: provider, connection, model, reason, expiresAt. Oper
## Other Resilience Features
- **15 routing strategies** (priority, weighted, round-robin, context-relay, fill-first, p2c, random, least-used, cost-optimized, reset-aware, strict-random, auto, lkgp, context-optimized) — see [AUTO-COMBO.md](../routing/AUTO-COMBO.md).
- **14 routing strategies** (priority, weighted, round-robin, context-relay, fill-first, p2c, random, least-used, cost-optimized, reset-aware, strict-random, auto, lkgp, context-optimized) — see [AUTO-COMBO.md](../routing/AUTO-COMBO.md).
- **Reset-aware routing** (v3.8.0) — prioritizes connections by quota reset time.
- **Background mode degradation** — Responses API `background: true` degraded to sync with warning.
- **Dynamic tool limit detection** — backs off providers when tool count limits hit.
- **Emergency fallback** — controlled by `OMNIROUTE_EMERGENCY_FALLBACK`; operators can override it from the Feature Flags page without a restart.
---
@@ -137,6 +141,22 @@ Provider-specific stealth (JA3/JA4, CCH, obfuscation) is separately documented
---
## Resilience testing (Fase 8 · Bloco C)
Além dos unit tests da lógica de resiliência, três testes exercitam o runtime sob
estresse/falha real (todos integração/nightly — nenhum bloqueia PR):
| Teste | O quê | Rodar |
|---|---|---|
| Chaos | Fake-upstream node injeta latência/reset/timeout/503 reais; valida que o circuit breaker abre/recupera e `checkFallbackError` classifica 503 como fallback recuperável. | `RUN_CHAOS_INT=1 npm run test:chaos` |
| Heap-growth | ~500 streams por `createSSEStream` sob `--expose-gc`; falha se o heap crescer além do teto (guarda OOM #3069). | `npm run test:heap` |
| k6 soak | Carga sustentada contra `/api/monitoring/health`; thresholds p95/erro. | `k6 run tests/load/k6-soak.js` (nightly) |
Orquestrados por `.github/workflows/nightly-resilience.yml` (cron + dispatch). No
`test:integration` default, chaos e heap se auto-skipam (sem `RUN_CHAOS_INT`/`--expose-gc`).
---
## See Also
- [Architecture Guide](./ARCHITECTURE.md) — System architecture and internals

48
Route-Guard-Tiers.md

@@ -1,5 +1,6 @@
> 🌍 [View in other languages](Languages)
# Route Guard Tiers
## Overview
@@ -22,10 +23,11 @@ non-loopback traffic would allow an attacker who obtained a valid JWT (e.g.,
via a Cloudflared/Ngrok tunnel) to trigger process spawning — a known CVE
class (GHSA-fhh6-4qxv-rpqj).
| Prefix | Reason | Bypassable by `manage`? |
| ------------------------- | -------------------------------------------------- | ----------------------- |
| `/api/mcp/` | MCP server — spawns stdio bridges and SSE handlers | Yes |
| `/api/cli-tools/runtime/` | CLI tool runtime — executes arbitrary plugin code | No (strict-loopback) |
| Prefix | Reason | Bypassable by `manage`? |
| ------------------------- | --------------------------------------------------------- | ----------------------- |
| `/api/mcp/` | MCP server — spawns stdio bridges and SSE handlers | Yes |
| `/api/cli-tools/runtime/` | CLI tool runtime — executes arbitrary plugin code | No (strict-loopback) |
| `/api/services/` | Embedded services (9router, CLIProxy) — npm install+spawn | No (strict-loopback) |
**Response on violation:** `403 LOCAL_ONLY`
@@ -39,9 +41,10 @@ default for any new LOCAL_ONLY path remains strict-loopback. Unauthenticated
requests and requests with non-manage keys are still rejected with
`403 LOCAL_ONLY`.
Today the only bypassable prefix is `/api/mcp/`. `/api/cli-tools/runtime/`
is intentionally excluded because it can spawn arbitrary subprocesses, which
is the exact CVE class the LOCAL_ONLY tier exists to prevent.
Today the only bypassable prefix is `/api/mcp/`. `/api/cli-tools/runtime/` and
`/api/services/` are intentionally excluded because they can spawn arbitrary
subprocesses (`npm install`, `node`), which is the exact CVE class the
LOCAL_ONLY tier exists to prevent.
| Request | Path | Result |
| ------------------------------------------- | -------------------------- | ------------------- |
@@ -130,6 +133,37 @@ expired DB doesn't silently downgrade to "deny".
| `tests/unit/authz/routeGuard.test.ts` | Unit tests for tier helpers |
| `tests/unit/authz/management-policy.test.ts` | Unit tests for evaluate() |
## Documenting Security Tiers in OpenAPI
When adding a new route to `docs/reference/openapi.yaml`, apply the corresponding
vendor extension if the route is classified by `routeGuard.ts`:
| routeGuard.ts classification | YAML annotation | Enforcement |
| ----------------------------- | -------------------------- | ----------------------------------------------- |
| `LOCAL_ONLY_API_PREFIXES` | `x-loopback-only: true` | Blocked from non-loopback unconditionally |
| `ALWAYS_PROTECTED_API_PATHS` | `x-always-protected: true` | Auth required even with `requireLogin=false` |
| Internal admin/debug route | `x-internal: true` | Hidden from /dashboard/api-endpoints by default |
| None (public / standard auth) | (no annotation needed) | Standard `requireLogin`-controlled access |
### Validation
Two scripts enforce consistency between YAML annotations and `routeGuard.ts`:
- `scripts/check/check-openapi-coverage.mjs` — fails if coverage < 99%
- `scripts/check/check-openapi-security-tiers.mjs` — fails if `x-loopback-only` or
`x-always-protected` annotations diverge from the compile-time constants
Both scripts run in the pre-commit hook and in CI.
### False Positive Rule
If `x-always-protected` or `x-loopback-only` is annotated on a route that is NOT in
the `routeGuard.ts` constant, the coverage script fails. The fix is always to align the
YAML to what `routeGuard.ts` actually enforces — not to add routes to `routeGuard.ts`
without also implementing the enforcement logic.
---
## See also
- `docs/security/CLI_TOKEN.md` — CLI machine-ID token

1
SQLite-Runtime.md

@@ -1,5 +1,6 @@
> 🌍 [View in other languages](Languages)
# SQLite Runtime Resolution
OmniRoute resolves its SQLite driver at startup through a 5-step fallback chain:

11
Setup-Guide.md

@@ -150,6 +150,15 @@ API Key: [copy from Endpoint page]
Model: if/kimi-k2-thinking (or any provider/model prefix)
```
If your editor cannot send `Authorization: Bearer ...`, use the tokenized compatibility base instead:
```txt
Base URL: http://localhost:20128/api/v1/vscode/YOUR_KEY/
Models URL: http://localhost:20128/api/v1/vscode/YOUR_KEY/models
Chat URL: http://localhost:20128/api/v1/vscode/YOUR_KEY/chat/completions
Ollama Tags URL: http://localhost:20128/api/v1/vscode/YOUR_KEY/api/tags
```
Works with Claude Code, Codex CLI, Gemini CLI, Cursor, Cline, OpenClaw, OpenCode, and OpenAI-compatible SDKs.
For detailed per-tool configuration (Claude Code, Codex CLI, Cursor, Cline, OpenClaw, Kilo Code, Copilot, and more), see the dedicated **[CLI Tools Guide](../reference/CLI-TOOLS.md)**.
@@ -255,7 +264,7 @@ For third-party Claude Code-compatible reverse proxies, OmniRoute keeps the defa
| `FETCH_CONNECT_TIMEOUT_MS` | `30000` | Undici TCP connect timeout |
| `FETCH_KEEPALIVE_TIMEOUT_MS` | `4000` | Undici idle keep-alive socket timeout |
| `TLS_CLIENT_TIMEOUT_MS` | inherits `FETCH_TIMEOUT_MS` | Timeout for TLS fingerprint requests made through `wreq-js` |
| `API_BRIDGE_PROXY_TIMEOUT_MS` | inherits `REQUEST_TIMEOUT_MS` or `30000` | Timeout for `/v1` proxy forwarding from API port to dashboard port |
| `API_BRIDGE_PROXY_TIMEOUT_MS` | inherits `REQUEST_TIMEOUT_MS` or `600000` | Timeout for `/v1` proxy forwarding from API port to dashboard port |
| `API_BRIDGE_SERVER_REQUEST_TIMEOUT_MS` | `max(API_BRIDGE_PROXY_TIMEOUT_MS, 300000)` | Incoming request timeout on the API bridge server |
| `API_BRIDGE_SERVER_HEADERS_TIMEOUT_MS` | `60000` | Incoming header timeout on the API bridge server |
| `API_BRIDGE_SERVER_KEEPALIVE_TIMEOUT_MS` | `5000` | Keep-alive timeout on the API bridge server |

200
Skills.md

@@ -4,7 +4,7 @@
# Skills Framework
> **Source of truth:** `src/lib/skills/` and `src/app/api/skills/`
> **Last updated:** 2026-05-13 — v3.8.0
> **Last updated:** 2026-05-28 — v3.8.6
OmniRoute exposes an extensible Skills framework that lets language models (and operators) compose reusable capabilities — from filesystem reads and HTTP requests to sandboxed code execution and curated marketplace skills.
@@ -12,6 +12,28 @@ A skill is a versioned, schema-defined unit of work. OmniRoute can inject skills
---
## Agent Skills vs Omni Skills
OmniRoute has two distinct but complementary skill systems:
| Dimension | **Omni Skills** (this doc) | **Agent Skills** |
| :--- | :--- | :--- |
| Purpose | LLM tool injection + sandboxed execution | SKILL.md catalog for external agents to discover and consume |
| Source of truth | `src/lib/skills/` + marketplace | `src/lib/agentSkills/` + `skills/` directory |
| Runtime mode | Injected into outbound requests, executed on tool-call events | Static markdown catalog + REST/MCP/A2A discovery endpoints |
| Who uses it | OmniRoute itself (combo routing, inbound LLM calls) | External agents, MCP clients, A2A orchestrators |
| Count | Variable (marketplace-driven) | 42 canonical entries (22 API + 20 CLI) |
| Format | `SkillDefinition` with tool schema + handler | `SKILL.md` frontmatter + markdown body |
| Discovery | `/api/skills/*` REST + `omniroute_skills_*` MCP tools | `/api/agent-skills/*` REST + `omniroute_agent_skills_*` MCP tools + A2A `list-capabilities` |
**Omni Skills** are the execution engine — they define what OmniRoute *can do* when an LLM invokes a tool.
**Agent Skills** are the documentation catalog — they explain to external agents *how to use* OmniRoute's REST API and CLI, with structured SKILL.md files that can be fed directly into agent prompts.
For the Agent Skills catalog, generator, MCP tools, and A2A skill, see [docs/frameworks/AGENT-SKILLS.md](./AGENT-SKILLS.md).
---
## Concepts
### Skill Sources
@@ -275,6 +297,182 @@ See [MCP-SERVER.md](./MCP-SERVER.md) for transport setup and scope assignments.
---
## Execution Lifecycle (v3.8.16+)
The `SkillExecutor` (`src/lib/skills/executor.ts`) is a **singleton** that manages every skill invocation. Understanding its lifecycle is critical for debugging timeouts, retries, and execution state.
### The 5-Stage Lifecycle
```
execute() called
│
▼
┌─────────────┐
│ PENDING │ ← queued, not yet started (DB row created)
└──────┬──────┘
│ start handler
▼
┌─────────────┐
│ RUNNING │ ← handler invoked with timeout
└──────┬──────┘
│
┌────┴────┬──────────┬──────────┐
│ │ │ │
▼ ▼ ▼ ▼
SUCCESS ERROR TIMEOUT (no other path — killed by parent)
│ │ │
└────┬────┴──────────┘
│
▼
DB row updated with status, output, durationMs
```
### Default Configuration
| Setting | Default | Configurable via |
|---------|---------|------------------|
| `timeout` | `30000` (30s) | `skillExecutor.setTimeout(ms)` |
| `maxRetries` | `3` | `skillExecutor.setMaxRetries(count)` |
> **Important**: The executor is a singleton — calling `setTimeout()` affects all subsequent invocations globally. Per-skill timeouts are not currently supported; if you need different timeouts per skill, submit separate processes or fork the executor.
### Status Values
From `src/lib/skills/types.ts`:
```ts
enum SkillStatus {
PENDING = "pending", // Queued, not yet started
RUNNING = "running", // Handler invoked
SUCCESS = "success", // Handler returned valid output
ERROR = "error", // Handler threw an exception
TIMEOUT = "timeout", // Exceeded the executor's timeout
}
```
> **Note**: The `TIMEOUT` status is defined in the enum but is **not actually written to the DB** by the current executor implementation — timeouts surface as `ERROR` with the message `"Skill execution timed out"`. The status enum is reserved for future use.
### Inspecting Executions
```ts
import { skillExecutor } from "omniroute/skills/executor";
// Get a specific execution by ID
const exec = skillExecutor.getExecution("exec-uuid-123");
if (exec) {
console.log(`${exec.skillName}: ${exec.status} in ${exec.durationMs}ms`);
}
// List recent executions for an API key
const recent = skillExecutor.listExecutions("api-key-id", 50, 0);
for (const e of recent) {
console.log(`${e.skillName} → ${e.status} (${e.durationMs}ms)`);
}
// Count total executions
const total = skillExecutor.countExecutions("api-key-id");
```
### Retry Behavior
The `maxRetries` setting is stored but **not currently used** by the executor's `execute()` method — it only performs a single attempt. The `maxRetries` value is exposed for future implementation and for hooks that want to read it.
For now, retries must be implemented inside the skill handler itself. Built-in
skills are registered against the executor (e.g. `registerBuiltinSkills(executor)`
/ `registerBrowserSkill(executor)` in `src/lib/skills/builtin/`); whichever handler
you register can wrap its own retry loop:
```ts
// inside a skill handler
async function handler(input, ctx) {
const maxRetries = 3;
let lastError: Error | null = null;
for (let attempt = 1; attempt <= maxRetries; attempt++) {
try {
return await fetchSomething(input);
} catch (err) {
lastError = err as Error;
if (attempt < maxRetries) {
await new Promise((r) => setTimeout(r, 1000 * attempt));
}
}
}
throw lastError;
}
```
---
## SkillMode in Detail
The `SkillMode` enum (`src/lib/skills/types.ts`) controls **when and how** skills are invoked:
```ts
enum SkillMode {
AUTO = "auto", // LLM decides when to call the skill
MANUAL = "manual", // Only invoked by explicit user request
HYBRID = "hybrid", // AUTO scoring + manual override
}
```
> **Note**: The codebase defines `SkillMode` (AUTO/MANUAL/HYBRID), while the `Skill.mode` field uses a different shape (`"on" | "off" | "auto"`). They are related but not identical — `SkillMode` is for executor policy, `Skill.mode` is for per-skill enablement.
### When to Use Each Mode
| Mode | LLM behavior | Use case |
|------|--------------|----------|
| `AUTO` | LLM can call the skill when it deems necessary | General-purpose skills (file reads, HTTP requests) |
| `MANUAL` | LLM cannot call the skill; only an explicit `executeSkill` API call invokes it | Sensitive operations (database writes, payments) |
| `HYBRID` | LLM can suggest the skill; user must confirm | Skills that have side effects but aren't dangerous |
### AUTO Scoring
When `AUTO` mode is active, each candidate skill is scored against the request
context by `scoreAutoSkill()` in `src/lib/skills/injection.ts` — an additive,
integer point system (skill-name match, name/tag/description token overlap,
background-reason hints, provider-hint bonus/penalty). The top
`AUTO_MAX_SKILLS = 5` skills with `score >= AUTO_MIN_SCORE = 3` are injected as
callable tools, ties broken by `installCount` then name. See the full point table
in [**Tool Schema Generation → AUTO Scoring**](#auto-scoring) earlier in this
document; there is no float `0.6`-style threshold and no `registry.ts` scoring.
---
## Built-in Skills Catalog
OmniRoute ships with a curated set of built-in skills in `src/lib/skills/builtin/`. The most common ones:
### Browser Automation Skill
The browser skill (`src/lib/skills/builtin/browser.ts`) provides headless browser automation via Playwright/Puppeteer. **It is implemented but not in the default skills catalog** — to use it, install the browser extension plugin separately.
```ts
// Enable in your config
const config: SkillConfig = {
enabled: true,
mode: SkillMode.MANUAL, // Always require explicit invocation
allowedSkills: ["browser"],
timeout: 60000, // 60s for page loads
maxRetries: 1,
};
```
### Other Built-in Categories
| Category | Skills | Mode |
|----------|--------|------|
| File I/O | `file_read`, `file_write` | AUTO |
| HTTP | `http_request` | AUTO |
| Search | `web_search` | AUTO |
| Code Exec | `eval_code` (sandboxed JavaScript/Python) | HYBRID |
| System | `execute_command` (sandboxed CLI execution) | MANUAL |
### Adding a Custom Skill
See the [Plugin SDK & Skills Integration](../plugins/PLUGIN_SDK.md) for how to add a custom skill via the plugin system.
---
## See Also
- [MCP-SERVER.md](./MCP-SERVER.md) — MCP tool registration and transports

8
Stealth-Guide.md

@@ -85,10 +85,10 @@ Applied to: `system` blocks, all `messages[].content`, and `tools[].description`
For third-party Anthropic relays that only accept "real Claude Code" traffic:
- `CLAUDE_CODE_COMPATIBLE_USER_AGENT = "claude-cli/2.1.146 (external, sdk-cli)"`
- `CLAUDE_CODE_COMPATIBLE_STAINLESS_PACKAGE_VERSION = "0.81.0"`
- `CLAUDE_CODE_COMPATIBLE_USER_AGENT = "claude-cli/2.1.158 (external, sdk-cli)"`
- `CLAUDE_CODE_COMPATIBLE_STAINLESS_PACKAGE_VERSION = "0.94.0"`
- `CLAUDE_CODE_COMPATIBLE_STAINLESS_RUNTIME_VERSION = "v24.3.0"`
- `anthropic-beta = "claude-code-20250219,interleaved-thinking-2025-05-14,effort-2025-11-24"`
- `anthropic-beta = "claude-code-20250219,interleaved-thinking-2025-05-14,effort-2025-11-24,redact-thinking-2026-02-12"`
- `CONTEXT_1M_BETA_HEADER = "context-1m-2025-08-07"` (Opus/Sonnet 4.x family)
- Default path: `/v1/messages?beta=true`
@@ -209,7 +209,7 @@ All MITM endpoints require management auth (`requireCliToolsAuth`). The sudo pas
| Variable | Default |
| ------------------------ | --------------------------------------------- |
| `CLAUDE_USER_AGENT` | `claude-cli/2.1.146 (external, cli)` |
| `CLAUDE_USER_AGENT` | `claude-cli/2.1.158 (external, cli)` |
| `CODEX_USER_AGENT` | `codex-cli/0.132.0 (Windows 10.0.26200; x64)` |
| `GITHUB_USER_AGENT` | `GitHubCopilotChat/0.45.1` |
| `ANTIGRAVITY_USER_AGENT` | `antigravity/2.0.1 darwin/arm64` |

27
Troubleshooting.md

@@ -3,12 +3,34 @@
# Troubleshooting
> **For Users**: Looking for quick fixes? See the [Quick Reference](#quick-reference) below.
🌐 **Languages:** 🇺🇸 [English](./TROUBLESHOOTING.md) | 🇧🇷 [Português (Brasil)](../i18n/pt-BR/docs/guides/TROUBLESHOOTING.md) | 🇪🇸 [Español](../i18n/es/docs/guides/TROUBLESHOOTING.md) | 🇫🇷 [Français](../i18n/fr/docs/guides/TROUBLESHOOTING.md) | 🇮🇹 [Italiano](../i18n/it/docs/guides/TROUBLESHOOTING.md) | 🇷🇺 [Русский](../i18n/ru/docs/guides/TROUBLESHOOTING.md) | 🇨🇳 [中文 (简体)](../i18n/zh-CN/docs/guides/TROUBLESHOOTING.md) | 🇩🇪 [Deutsch](../i18n/de/docs/guides/TROUBLESHOOTING.md) | 🇮🇳 [हिन्दी](../i18n/in/docs/guides/TROUBLESHOOTING.md) | 🇹🇭 [ไทย](../i18n/th/docs/guides/TROUBLESHOOTING.md) | 🇺🇦 [Українська](../i18n/uk-UA/docs/guides/TROUBLESHOOTING.md) | 🇸🇦 [العربية](../i18n/ar/docs/guides/TROUBLESHOOTING.md) | 🇯🇵 [日本語](../i18n/ja/docs/guides/TROUBLESHOOTING.md) | 🇻🇳 [Tiếng Việt](../i18n/vi/docs/guides/TROUBLESHOOTING.md) | 🇧🇬 [Български](../i18n/bg/docs/guides/TROUBLESHOOTING.md) | 🇩🇰 [Dansk](../i18n/da/docs/guides/TROUBLESHOOTING.md) | 🇫🇮 [Suomi](../i18n/fi/docs/guides/TROUBLESHOOTING.md) | 🇮🇱 [עברית](../i18n/he/docs/guides/TROUBLESHOOTING.md) | 🇭🇺 [Magyar](../i18n/hu/docs/guides/TROUBLESHOOTING.md) | 🇮🇩 [Bahasa Indonesia](../i18n/id/docs/guides/TROUBLESHOOTING.md) | 🇰🇷 [한국어](../i18n/ko/docs/guides/TROUBLESHOOTING.md) | 🇲🇾 [Bahasa Melayu](../i18n/ms/docs/guides/TROUBLESHOOTING.md) | 🇳🇱 [Nederlands](../i18n/nl/docs/guides/TROUBLESHOOTING.md) | 🇳🇴 [Norsk](../i18n/no/docs/guides/TROUBLESHOOTING.md) | 🇵🇹 [Português (Portugal)](../i18n/pt/docs/guides/TROUBLESHOOTING.md) | 🇷🇴 [Română](../i18n/ro/docs/guides/TROUBLESHOOTING.md) | 🇵🇱 [Polski](../i18n/pl/docs/guides/TROUBLESHOOTING.md) | 🇸🇰 [Slovenčina](../i18n/sk/docs/guides/TROUBLESHOOTING.md) | 🇸🇪 [Svenska](../i18n/sv/docs/guides/TROUBLESHOOTING.md) | 🇵🇭 [Filipino](../i18n/phi/docs/guides/TROUBLESHOOTING.md) | 🇨🇿 [Čeština](../i18n/cs/docs/guides/TROUBLESHOOTING.md)
Common problems and solutions for OmniRoute.
---
## Quick Reference
**New to OmniRoute?** Start here — these solve 90% of problems:
| I see this | What it means | What to do |
|------------|--------------|------------|
| "Can't connect" | OmniRoute isn't running | Run `omniroute` or `docker restart omniroute` |
| "Invalid API key" | Your key is wrong or expired | Re-copy the key from the provider's website |
| "Rate limit exceeded" | You're sending too many requests | Wait 1 minute, or use `model: "auto"` for automatic fallback |
| "Quota exceeded" | You've used up your free/paid quota | Connect more providers, or use free providers (Kiro, Pollinations) |
| "Slow responses" | Provider is busy or far away | Use `model: "auto/fast"` or connect a faster provider (Groq, Cerebras) |
| "Wrong provider used" | `auto` picked a different provider | That's normal! `auto` picks the best one. Force a specific provider with `model: "openai/gpt-4o"` |
| "502 Bad Gateway" | Provider is down | Wait and retry, or use `model: "auto"` to switch providers |
| "401 Unauthorized" | Your credentials are wrong | Check your API key or re-authenticate with OAuth |
| "429 Too Many Requests" | Rate limited | Wait 1 minute, or connect more providers |
**Still stuck?** See the [Quick Fixes](#quick-fixes) below, or ask on [Discord](https://discord.gg/hmexnhgE).
---
## Quick Fixes
| Problem | Solution |
@@ -149,7 +171,7 @@ To gain isolation, delete the old connection from Dashboard → Providers and re
via any of the three import flows.
For full details and step-by-step instructions for adding two Kiro accounts side by side,
see [`docs/guides/KIRO_SETUP.md`](./KIRO_SETUP.md).
see [`docs/guides/KIRO_SETUP.md`](../guides/KIRO_SETUP.md).
---
@@ -234,6 +256,9 @@ curl http://localhost:20128/api/monitoring/health
- Application logs: `<repo>/logs/...` (when `APP_LOG_TO_FILE=true`)
- Call log artifacts: `${DATA_DIR}/call_logs/YYYY-MM-DD/...` when the call log pipeline is enabled
The Request Logs page's **Clean history** action clears `call_logs`, legacy
`request_detail_logs`, and the local `${DATA_DIR}/call_logs/` artifact directory.
---
## Circuit Breaker Issues

29
User-Guide.md

@@ -141,6 +141,10 @@ Models:
**Pro Tip:** Use Opus for complex tasks, Sonnet for speed. OmniRoute tracks quota per model!
Claude and Claude Code-compatible routes preserve `max` thinking effort for Opus and Sonnet
models. Haiku models do not accept the `max` effort tier, so OmniRoute downgrades that
request to a high thinking budget before sending it upstream.
#### OpenAI Codex (Plus/Pro)
```bash
@@ -575,7 +579,7 @@ For the full environment variable reference, see the [README](../README.md).
> The list below is curated from `open-sse/config/providerRegistry.ts` for v3.8.0. Cloud catalogs (Gemini, OpenRouter, etc.) are synced dynamically — for the full live catalog open **Dashboard → Providers → [provider] → Available Models** or call `GET /api/models/catalog`.
**Claude Code (`cc/`)** — Pro/Max OAuth: `cc/claude-opus-4-7`, `cc/claude-opus-4-6`, `cc/claude-opus-4-5-20251101`, `cc/claude-sonnet-4-6`, `cc/claude-sonnet-4-5-20250929`, `cc/claude-haiku-4-5-20251001`
**Claude Code (`cc/`)** — Pro/Max OAuth: `cc/claude-opus-4-8`, `cc/claude-opus-4-7`, `cc/claude-opus-4-6`, `cc/claude-opus-4-5-20251101`, `cc/claude-sonnet-4-6`, `cc/claude-sonnet-4-5-20250929`, `cc/claude-haiku-4-5-20251001`
**Codex (`cx/`)** — Plus/Pro OAuth: `cx/gpt-5.5` (+ effort tiers: `gpt-5.5-xhigh`, `gpt-5.5-high`, `gpt-5.5-medium`, `gpt-5.5-low`), `cx/gpt-5.4`, `cx/gpt-5.4-mini`, `cx/gpt-5.3-codex`, `cx/gpt-5.3-codex-spark`, `cx/gpt-5.2`
@@ -623,7 +627,7 @@ For the full environment variable reference, see the [README](../README.md).
**Other compatible providers** (selected): `cohere`, `databricks`, `snowflake`, `together`, `vertex`, `alibaba`, `alibaba-cn`, `bedrock` (via `aws-bedrock`), `azure-ai`, `openrouter` (passthrough catalog), `siliconflow`, `hyperbolic`, `huggingface`, `featherless-ai`, `cloudflare-ai`, `scaleway`, `deepinfra`, `vercel-ai-gateway`, `bazaarlink`, `friendliai`, `nous-research`, `reka`, `volcengine`, `ai21`, `gigachat`. Each maintains its own model list in `providerRegistry.ts` and can be auto-synced when the provider exposes a `/models` endpoint.
**Note on model IDs:** OmniRoute uses provider-native IDs (`claude-opus-4-7`, `gpt-5.5`, `glm-5.1`, `MiniMax-M2.7`, `kimi-k2.5`, `grok-4.20-0309-reasoning`). Some IDs include dotted versions because that is how the upstream API expects them. If a model is not listed above, run `omniroute models --search <term>` or hit `GET /api/models/catalog` to confirm availability.
**Note on model IDs:** OmniRoute uses provider-native IDs (`claude-opus-4-8`, `gpt-5.5`, `glm-5.1`, `MiniMax-M2.7`, `kimi-k2.5`, `grok-4.20-0309-reasoning`). Some IDs include dotted versions because that is how the upstream API expects them. If a model is not listed above, run `omniroute models --search <term>` or hit `GET /api/models/catalog` to confirm availability.
</details>
@@ -819,10 +823,12 @@ OmniRoute implements provider-level resilience with five components:
- **Use Upstream Retry Hints** — Honors authoritative `Retry-After` or reset hints when provided
- **Max Backoff Steps** — Maximum exponential backoff level for repeated failures
3. **Provider Circuit Breaker** — Tracks end-to-end provider failures and automatically opens the breaker when the configured threshold is reached:
- **Failure Threshold** — Consecutive provider failures before opening the breaker
3. **Provider Circuit Breaker** — Tracks end-to-end provider failures, marks a provider degraded at the configured warning threshold, and opens the breaker when the configured failure threshold is reached:
- **Degradation Threshold** — Consecutive provider failures before entering `DEGRADED`
- **Failure Threshold** — Consecutive provider failures before entering `OPEN`
- **Reset Timeout** — Time window before the provider is tested again
- **CLOSED** (Healthy) — Requests flow normally
- **DEGRADED** — Requests still flow while elevated failures are tracked
- **OPEN** — Provider is temporarily blocked after repeated failures
- **HALF_OPEN** — Testing if provider has recovered
@@ -970,6 +976,21 @@ Configure per-combo balancing in **Dashboard → Combos → Create/Edit → Stra
| **Cost-Optimized** | Routes to the cheapest available model (uses pricing table) |
Global combo defaults can be set in **Dashboard → Settings → Routing → Combo Defaults**.
Combo target timeouts inherit the current request timeout by default. Use **Target timeout
(seconds)** on combo defaults or an individual combo only when a shorter per-target limit should
trigger faster fallback.
Zero-latency combo optimizations are opt-in. Leave **Zero-latency optimizations** disabled to
prevent these latency features from racing fallback targets, skipping targets based on TTFT
history, or compressing fallback requests; enabling it allows configured hedging, predictive TTFT
skips, and proactive fallback compression to trade routing/request fidelity for lower tail
latency.
Disable **Reasoning token buffer** when upstream providers require strict
`max_tokens` / `maxOutputTokens` limits. When enabled, combo routing only adds reasoning-model
headroom for models with a known output cap and leaves the client token limit unchanged when the
safe buffered value would exceed that cap. If the client limit is already above a known cap,
OmniRoute clamps it down to that cap before sending the upstream request.
---